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Introduction: 


There  is  great  uncertainty  about  how  to  best  screen  for  and  treat  prostate  cancer 
Randomized  tnals  of  screening  and  treatment  are  underway,  but  will  take  years  to 
complete.  In  the  meantime,  patients  and  their  doctors  face  difficult  decisions.  For  these 
reasons,  decision  analysis  has  been  widely  used  to  study  the  costs  and  effectiveness  of 
screening  and  treatment  for  prostate  cancer.  We  developed  a  comprehensive  model  of  the 
natural  history, screening,  and  treatment  of  prostate  cancer  based  on  pathologic  stage,  grade 
of  tumor,  and  other  important  aspects  neglected  by  previous  decision  analysis  Model 
parameters  are  derived  by  extensive  literature  review  and  validated  using  aggregate  and 
individual-level  SEER  data.  We  also  extended  the  model  to  assess  the  effectiveness,  costs 
and  cost-effectiveness  of  screening  and  treatment  policies  from  age  50  to  99.  Our  baseline’ 
model  suggests  that  screening  and  treatment  improve  non-discounted  life  expectancy  for 
men  ages  50-70  with  more  advanced  tumors,  but  offer  only  modest  increase  for  well- 
lfferentiated  tumors.  When  quality  of  life  weights  were  considered,  quality-adjusted  life 
expectancy  was  reduced.  The  more  advanced  sensitivity  analysis  is  in  progress  with 
funding  from  the  National  Cancer  Institute. 


Body: 


As  specified  in  the  Statement  of  Work  section  of  our  original  proposal,  all  the 
goals  are  accomplished  in  line  with  the  time  schedule. 

Overall,  our  model  is  comprised  of  three  key  features.  The  first  is  a  model  of  the 
natural  history  of  prostate  cancer.  This  consists  of  a  Markov  process  defined  by  a  state 
vector  that  allows  us  to  carry  states  of  symptoms  and  stages  from  year  to  year  and  a 
transition  matrix  that  describes  the  progression  of  prostate  cancer.  A  decision  tree  is 
developed  to  model  the  probability  that  a  man  has  prostate  cancer  detected  in  a  year  given 
the  disease  state  that  he  carries  from  the  beginning  of  the  year  (see  Appendix  1).  The 
second  is  the  solid  parameter  justification.  Model  parameters  are  estimated  by  extensive 
literature  review,  second  data  analysis  and  mathematical  modeling.  We  use  maximum 
likelihood  approach  to  validate  the  parameters  using  aggregate  incidence  and  mortality 
data  as  well  as  individual  level  data.  The  third  is  estimation  of  cost-  effectiveness  with 
various  screening  policies  and  treatment  policies. 

We  will  report  our  progress  in  relation  to  the  Statement  of  Work  below. 

Task  1.  Review  of  model  structure  to  reflect  evolving  innovation,  Months  1-3. 

a)  Relevance  given  current  knowledge  of  prostate  biology  (i.e.  grade,  p53  early 

metastases)  ’  J 

b)  Relevance  given  current  screening  and  treatment  technology  (i.e.  PSA  density 
%  free  PSA) 

c)  Relevance  for  potential  future  technologies  (i.e.  PCR,  Indium-1 1 1  labeled 
antibody  to  detect  metastases,  gene  therapy) 

One  important  feature  in  our  model  is  a  state  vector  containing  elements  to  carry 
rom  year  to  year.  To  determin  what  biological  aspects  are  necessary  to  incorporate  in  the 
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state  vector,  we  have  conducted  extensive  literature  review  on  the  biology  of  prostate 
cancer.  Although  prostate  cancer  has  often  been  characterized  as  a  slow-growing  tumor 
the  natural  history  of  prostate  cancer  varies  tremendously  based  on  tumor  grade  and 
stage.  ^  Therefore,  we  have  included  prostate  cancer  grade  in  our  state  vector  that  we 
carry  from  year  to  year.  Our  comprehensive  literature  review  has  helped  us  determine  the 
probabilities  that  a  man  transitions  among  states. 

Although  the  identification  of  p53,  a  tumor  suppressor  gene,  has  been  found  to  be 
one  important  advance  in  prostate,  its  utilization  in  standard  practice  is  premature  to  the 
point.  Before  its  prognostic  use  is  confirmed  by  future  studies,  we  have  not  included 
information  concerning  p53  into  our  decision  analysis.  However,  our  model  is  flexibly 
programmed  to  permit  the  inclusion  of  new  development  concerning  p53. 

Prostate  specific  antigen(PSA)  testing  has  diagnostic  and  prognostic  value  in 
practice.  PSA  test  remains  as  the  most  relevant  technology  for  detecting  prostate  cancer.  It 
is  also  mcreasmly  popular  in  clinical  staging  of  prostate  cancer  tumors.  Therefore,  we  have 
incorporated  PSA  level  detectability  as  a  element  in  the  state  vector  of  our  model. 

However,  the  benefits  offered  by  PSA  density  remain  to  be  further  explored.4 
Likewise,  the  data  is  still  too  limited  to  determine  whether  free  PSA  is  clinically  useful.5,6,7 
As  such,  we  have  not  formally  incorporated  such  measurement  techniques  into  the  model 
but  our  model  is  flexible  enough  to  include  those  techniques  if  their  diagnostic  or 
prognostic  value  are  confirmed. 

Our  comprehensive  literature  review  indicates  none  of  the  new  therapies  listed 
above  (PCR,  Indium- 1 1 1  labeled  antibody  to  detect  metastases,  gene  therapy)  have  come 
mto  widespread  use  as  diagnostic  or  treatment  tools  for  men  with  prostate  cancer. 

However,  The  flexibility  of  our  model  allows  us  to  include  the  effects  of  new  diagnostic 
and  therapies  on  prostate  cancer  detection,  treatment,  and  outcomes. 

Task  2.  Programming  of  model  to  reflect  revised  structure,  Months  1-6 

a)  Revision  of  decision  tree  if  needed 

b)  Revision  of  natural  history  model  if  needed 

c)  Programming  of  costs 

d)  Programming  of  benefits  and  quality  of  life  adjustments 

We  have  refined  our  decision  tree  substantially  to  make  our  model  more  accurate 
compared  to  the  original  one  in  our  proposal.  For  example,  in  the  original  model  we 
assume  a  given  percentage  of  men  would  have  DRE  detectable  prostate  cancer.  However 
because  men  who  do  not  have  DRE  detectable  prostate  cancer  are  less  likely  to  have  it  the 
next  year,  the  original  approach  would  have  overestimated  the  cumulative  number  of  men 
who  have  DRE  detectable  prostate  cancer.  Therefore,  the  likelihood  that  men  would  have 
rectal  detectable  cancer  should  be  conditional  on  whether  or  not  symptoms  were  detected 
in  the  previous  year.  To  address  the  complication  of  progression  of  prostate  cancer  we 
have  created  a  large  matrix  with  528  states  to  carry  over  year.  The  states  incorporate 
combinations  of  cancer  symptom(two  states),  screening  status(two  states),  BPH 
s>mptoms(two  states),  metastatic  detectability(two  states),  rectal  and  PSA  detectability 
(six  states),  having  had  a  TURP(two  states),  true  state  (five  states),  and  true  grade(three 
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states).  The  probability  of  moving  into  each  of  the  states  is  conditional  on  the  state  in 
previous  year. 

We  have  designed  our  model  to  be  flexible  so  that  we  can  allow  for  the  changes  in 
costs,  benefits,  and  quality  of  life.  We  have  conducted  a  number  of  sensitivity  analyses 
concerning  quality  of  life,  relecting  the  difference  of  utility  obtained  through  specific 
treatment.  Orgimally,  we  incorporate  the  anxiety  into  our  model  as  a  main  complication  of 
watchful  waitng  treatment.  The  results  indicate  the  potential  gains  can  be  large  if  the 
patients  experience  substantial  anxiety. 

Task  3.  Review  of  literature  to  estimate  model  parameters,  Months  4-18 

a)  Identification  of parameters  of  interest 

b)  Collection  of  articles  relevant  for  parameters  of  interest 

c)  Review  and  analysis  of  articles 

d)  Preparation  of  documentation  of  literature  review 


We  have  refined  the  decision  tree  model  to  identify  the  parameters  of  importance 
We  estimate  each  model  parameter  via  one  of  the  following  approaches:  1)  comprehensive 
literature  review;  2)analysis  of  primary  or  secondary  data;  3)mathematical  modeling.  For 
example,  to  estimate  the  probability  that  a  biopsy  will  be  positive  in  the  presence  of  stage 
A  cancer,  we  design  a  geometric  model  of  a  needle  transversing  a  hypothetical  prostate  of 
a  standard  volume  encountering  a  tumor  of  specified  volume  present  in  the  prostate  We 
also  validate  the  parameter  with  post-mortem  studies  of  biopsy  yield  in  men  examined  for 
prostate  cancer  on  autopsy.  The  complete  discussion  on  our  parameter  justification  is  in 
the  paper  "Patterns  of  prostate  cancer  treatment  by  clinical  stage  and  age  in  the  United 
States,"  which  has  been  published  in  the  American  Journal  of  Public  Health. 

Our  extensive  literature  review  covers  many  carefully  performed  analysis  of 
prostate  cancer  screening  and  treatment.8'9’10’1 1’12>13’14>15>16’17-18>r9-20  Although  most  of  the 
papers  have  many  strengths,  all  of  them  have  major  flaws  undermining  their  validity  and 
relevance.  The  first  and  most  important  flaw  is  that  those  studies  fail  to  model  prostate 
cancer  incidence  as  a  dynamic  process  that  may  be  affected  by  previous  screening  patterns 
The  second  major  limitation  of  existing  model  is  that  they  fail  to  distinguish  either  between 
c  mical  stage  and  pathological  stage,  or  between  cancers  of  differing  stage  and  grade.  The 
existing  literature  has  also  minor  flaws,  including  ignoring  the  sensitivity/specificity  of 
screening  tests,  failing  to  consider  quality  of  life,  and  a  variety  of  data-related  issues. 

Conclusively,  based  on  the  existing  models,  we  have  developed  more  complicated 
model  to  account  for  extra  key  aspects  of  the  natural  history,  screening,  and  treatment  of 
prostate  cancer.  As  highlighted  in  our  annual  report,  those  factors  are  to  model: 

(1) .  Tumor  heterogeneity  and  progression  rates  by  grade. 

(2) .  Misclassification  of  tumor  stage  due  to  the  discrepancy  between  clinical  and 
pathologic  stage. 


(3) .  The  ability  of  screening  and  treatment  to  affect  the  prevalence  of  prostate 

(4) .  The  effect  of  BPH  and  other  prostate  symptoms  and  their  management  on 
of  prostate  cancer. 


cancer, 
the  detection 


(5).  The  potential  effects  of  screening  and  treatment  of  quality  of  life,  as  measured  by 
quality-adjusted  life  expectancy. 
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Task  4:  Sensitivity  Analysis:  Months  6-18 

a)  All  parameters  in  model 

b)  One  and  multi-way  sensitivity  analysis 

c)  Assess  effects  on: 

1)  Costs 

2)  Effectiveness 

3)  Cost-effectiveness 


a  ■  ha,Ve  comPleted  the  sensitivity  analyses  regarding  to  the  parameter  estimates 
derived  from  literature  review,  meta-analyses,  and  secondary  data  analysis.  Our  model 
a  ows  for  three  levels  of  parameters,  which  are  the  best  estimate,  the  upper  and  lower 
boundaries  of  the  parameters.  But  some  parameters,  such  as  quality  of  life  weights  have 
no  analytical  bounds.  Therefore,  we  use  the  range  of  quality  of  life  weights  in  the  ’ 
literature  for  sensitivity  analysis.  Finally,  we  employ  the  maximum  likelihood  approach  to 
optimize  our  decision  model.  However,  it  is  a  very  time-consuming  task  which  can  take 
months  to  achieve  the  optimization.  If  one  parameter  or  assumption  is  modified,  the  whole 
process  has  to  run  over  again.  The  final  optimized  result  is  presented  in  Appendix  2 
More  detailed  discussion  is  in  Task  5. 


We  have  conducted  extensive  research  on  cost-effectiveness  analysis.  We  adopt 
the  expected  utility  maximization  approach  to  perform  the  sensitivity  analysis.  Generally 
speakmg,  utility  assessment  reflects  the  strength  of  the  preference  or  the  degree  of 
abhorrence  for  the  potential  outcome  of  interest.  Our  work  focuses  on  determining  patient 
utilities  for  different  states  with  prostate  cancer  and  treatment  complications  for  the  cost- 
effectiveness  analysis.  The  output  of  our  research  is  to  date  a  series  of  publications  on 
cost-effectiveness  analysis,  as  listed  in  the  Reportable  Outcomes  section  below,  and 
more  publications  are  in  progress. 

•  f  Anoth^.ongmal  contribution  to  the  existing  literature  is  to  incorporate  future  costs 
into  the  cost-effectiveness  analysis.  We  have  published  a  number  of  papers2 1>22’23 

indicating  the  importance  of  including  future  medical  and  non-medical  costs  in  cost- 
effectiveness  analysis.  Simply  put,  we  add  costs  associated  with  non-prostate  cancer 
re  ated  medical  expenditures  and  non-medical  consumption  net  of  earnings  to  total  costs 
accumulated  m  each  years  of  life.  Theoretically,  the  inclusion  of  future  costs  will  most 
adversely  affect  screening  and  treatment  that  have  relative  small  effects  on  quality-adjusted 
hfe  expectancy  compared  to  life  expectancy.  Therefore,  the  cost-effectiveness  analysis 

wi  future  costs  will  favor  the  interventions  that  minimize  complications  of  treatment  for 
prostate  cancer. 


Task  5:  Estimation  of  Transition  Rates  Using  SEER  and  Watchful  Waiting  Data,  Months 

a)  Programming  of  estimation  techniques 

b)  Data  cleaning 

c)  Estimation 

d)  Sensitivity  Analyses 

As  mentioned  in  Task  3,  the  most  important  contribution  of  our  decision  model  is 
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to  simulate  the  prostate  cancer  incidence  as  a  dynamic  process  that  is  neglected  by  most 
existing  models.  The  key  parameters  of  our  model  are  the  transition  probabilities  between 
stage,  grade  and  death.  Although  we  can  estimate  most  parameters  of  interest  via  literature 
review  or  data  analysis,  there  is  no  information  available  concerning  the  transition 
probability.  Therefore,  we  adopt  the  curve  fitting  technique  to  estimate  the  transition 
matrix.  Specifically,  the  constrained  optimization  is  introduced  to  reduce  the  sum  of 
squares  difference  between  our  predicted  aggregate  incidence  and  mortality  data  and  SEER 
data.  As  can  be  seen  in  the  Appendix  2,  the  predicted  mortality  results  and  clinical  stage 
results  fit  the  SEER  results  very  well,  though  not  perfectly.  Because  of  the  time  limitation 
and  computing  power  limitation,  it  is  extremely  difficult  to  completely  reduce  the  sum  of 
squares  to  zero.  We  have  received  R01  funding  from  the  National  Cancer  Institute  to 
continue  our  optimization  in  the  aggregate  level.  In  the  meantime,  we  will  extend  the 
optimization  to  the  individual  level,  i.e.,  optimizing  the  maximum  likelihood  function 
constructed  by  the  individual  data.  We  will  determine  the  analytic  probability  that  a 
person  in  SEER  will  have  an  observed  detection  pattern,  treatment  pattern,  and  mortality 
pattern  according  to  our  decision  model. 

Task  6:  Preparation  of  Papers  Reporting  Cost-Effectiveness  Analyses,  Months  18-30 

a)  Evaluation  of  established  existing  technologies 

1) Treatment,  esp.  of  special  patient  subgroups 

2)  Rectal 

3)  PSA 

4)  Differing  screening  ages  and  intervals 

b)  Evaluations  of  innovative  technologies  and  management  strategies 

1)  Percent  Free-PSA 

2)  Detection  of  micrometastases 

3)  Other  technologies 

We  have  completed  preliminary  cost-effectiveness  analysis  based  on  treatment  and 
age  primarily.  The  detailed  results  are  presented  in  Appendix  3. 

Table  1  a-3  a  in  Appendix  3  provide  the  baseline  cost-effectiveness  results  of  age- 
specific  treatments  for  clinically  localized  well,  moderate  and  poor  grade  cancers 
respectively.  The  baseline  analysis  uses  a  discount  rate  of  3%  and  incorporates  the 
baseline  QOL  weights  of  impotence,  incontinence,  urethral  stricture,  bowel  dysfunction 
and  metastasis  to  compute  the  quality-adjusted  life-years  (QALYs).  There  are  some 
noticeable  increases  in  non-discounted  life  years  with  treatments  (radiation  or  surgery) 
over  watchful  waiting  especially  at  younger  ages  over  all  grades  of  cancer.  Moreover,  the 
cost-effectiveness  ratio  of  cost  per  discounted  life  years  for  treatments  over  watchful 
waiting  is  cost-effective  by  the  $100K/Life  Years  criteria  at  younger  ages  over  all  grades 
of  cancer.  However,  once  QOL  weights  are  included  in  the  calculations,  all  treatments 
become  dominated  by  watchful  waiting  at  all  ages  and  all  grades  of  cancer.  The  results 
stress  the  importance  of  patients’  perception  of  side  effects  as  an  important  factor  in 
choosing  treatment. 

The  results  are  found  to  be  highly  sensitive  to  psychological  anxiousness  of  the 
patient  detected  with  prostate  cancer.  No  literature  could  be  identified  that  studies  the 
effect  of  such  anxiety  on  the  quality  of  life  of  patients.  Moreover,  this  anxiety  is  expected 
to  change  with  treatment  if  the  cancer  could  be  eliminated.  We  assigned  a  new  QOL 
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weight  to  patient  with  anxiety.  If  patients  are  detected  with  prostate  cancer  their  baseline 
QOL  is  equated  to  this  anxiety  QOL  in  the  absence  of  any  other  side  effects.  If  they  are 
treated  with  surgery  we  assume  that  the  cancer  could  be  removed  with  surety  and  the 
patient’s  QOL  will  become  1  (normal)  in  the  absence  of  any  other  side  effects.  In  case  of 
radiation,  though  the  cancer  may  be  fully  cured  there  is  no  way  to  ascertain  this  for  sure 

and  hence  the  patient’s  QOL  is  given  a  weight  =  (1+  Anxiety  QOL)/2  in  the  absence  ofay 
other  side  effects.  y 

Table  lb  -3b  shows  the  sensitivity  analysis  of  anxiety  QOL  on  the  cost- 
effectiveness  results.  Modest  increase  in  anxiety  (or  modest  decrease  in  anxiety  QOL) 
leads  to  favorable  cost-effectiveness  ratios  for  radiation  and  surgery.  Such  effects  appears 
o  e  true  over  all  ages  and  all  grades  of  cancer,  though  younger  patients  with  more 
advanced  grade  of  cancer  seem  to  benefit  most  from  treatments  if  they  suffer  from  anxiety. 

.  grant  from  the  National  Cancer  Institute  will  allow  us  to  extend  the  cost- 

etfectiveness  under  vanous  screening  policies,  innovative  technologies,  and  management 


Key  Research  Accomplishments 

•  Created  more  accurate  model  concerning  the  natural  history  of  prostate  cancer  and  a 
more  accurate  decision  tree  concerning  the  treatment  of  prostate  cancer. 

•  Identified  relevant  natural  history,  treatment,  and  outcome  parameters  for  inclusion  in 
our  natural  history  and  decision  tree  models. 

Determined  the  model  parameter  probabilities  using  literature  reviews,  meta-analvses 
and  primary  data  analysis.  ’ 

Completed  preliminary  cost-effectiveness  analyses  that  indicate  that  treatment  is  cost- 
effective  in  terms  of  life  years. 

Completed  preliminary  cost-effectiveness  analyses  that  indicate  that  the  cost-effective 
ratio  is  sensitive  to  QALYs  resulting  from  treatment  decisions  at  younger  ages  and  the 
anxiety  QOL  at  all  ages. 

•  Published  a  peer-reviewed  journal  paper  describing  prostate  cancer  treatment  rates 
across  clinical  stages. 

•  Published  a  peer-reviewed  journal  paper  describing  the  theoretical  basis  for  addressing 
uncertainty  in  cost-effectiveness  analysis 

•  Have  almost  completed  a  major  paper  examining  the  cost-effectiveness  of  prostate 
cancer  treatment.  Already  presented  in  abstract  form  at  the  Society  for  Medical 
Decision  Making. 

Received  R01  funding  from  the  National  Cancer  Institute  to  continue  this  work  to  final 
analysis  of  cost-effectiveness 

(8)  Reportable  Outcomes 

Grants  and  Fellowships: 
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♦Cost-Effectiveness  of  Prostate  Cancer  Screening  and  Treatment.  National  Cancer 
Institute,  09/01/01-  08/31/03,  $1,142,433.  R01-CA92443. 

♦John  M.  Olin  Foundation  Faculty  Fellowship,  7/1/99/10/1/00,  $108,917. 

Published  Articles: 

♦Meltzer  D,  Egleston  BL,  Abdalla  I.  Patterns  of  Prostate  Cancer  Treatment  by  Clinical 
Stage  and  Age.  Aermican  Journal  of  Public  Health.  91(1):  126-128,  2001. 

♦Meltzer  D.  Addressing  Uncertainty  in  Medical  Cost-Effectiveness  Analysis:  Implications 
of  Expected  Utility  Maximization  for  Methods  to  Perform  Sensitivity  Analysis  and  the 
Use  of  Cost-Effectiveness  Analysis  to  Set  Priorities  for  Medical  Research.  Journal  of 
Health  Economics,  20( 1 ) :  1 09- 1 29,  200 1 . 

Meltzer  D,  Johannesson  M.  Inconsistencies  in  the  ‘Societal  Perspective’  on  Costs  of  the 
Panel  on  Cost-Effectiveness  in  Health  and  Medicine.  Medical  Decision  Makins 
19:371-377,1999 

♦Meltzer  D,  Johannesson  M.  On  the  Role  of  Theory  in  Cost-Effectiveness  Analysis:  A 
Response  To  Garber,  Russell,  and  Weinstein.  Medical  Decision  Making,  19:383- 


Published  Abstracts 


♦Meltzer  D,  Hazen  G,  Johanesson  M,  Abdalla  I,  Chang  R,  Elstein  A,  Schwartz  A.  Effect 
of  Future  Costs  on  the  Cost-Effectiveness  of  Life  Extension  and  Quality  of  Life 
Improvement  Among  the  Elderly.  Medical  Decision  Making_ 18:  475,  October  1998. 

♦Meltzer  D,  Polonsky  T.  Do  Quality- Adjusted  Life  Years  Reflect  Patient  Preferences? 
Validation  Using  Revealed  Preference  for  Intensive  Treatment  of  Insulin-Dependent 
Diabetes  Mellitus.  Medical  Decision  Making  18:  459,  October  1998. 

♦Meltzer  D,  Basu  A,  Egleston  B.  Structure  and  Early  Results  of  a  Prostate  Cancer  Decision 
Model.  Medical  Decision  Making.  In  press. 

Presentations: 

“Implications  of  Expected  Utility  Maximization  for  Assessing  Cost-Effectiveness  under 
Uncertainty:  Methods  for  Sensitivity  Analysis  and  Implications  for  the  Use  of  Cost- 
Effectiveness  Analysis  to  Set  Priorities  for  Medical  Research”.  National  Bureau  of 
Economic  Research  Summer  Institute,  Boston,  MA,  July  1998. 

Effect  of  Future  Costs  on  the  Cost-Effectiveness  of  Life  Extension  and  Quality  of  Life 
Improvement  Among  the  Elderly",  Society  for  Medical  Decision  Making  Annual 
Meeting,  Plenary  Address,  Boston,  MA,  October  1998. 
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"Accounting  for  Future  Costs  from  a  Societal  Perspective  in  Cost-Effectiveness  Analysis": 
Plenary  Address,  International  Health  Economics  Association  Annual  Meeting 
Rotterdam,  Netherlands,  June  1999. 

“Can  Burden  of  Illness  Measures  Aid  Priority-setting  at  NIH?”  Invited  Presentation  to  Df. 
Harold  Varmus  and  panel  of  experts  convened  at  NIH  in  Response  to  IOM  Report 
on  Scientific  Priorities  at  NIH,  June  1999. 

“Measuring  Quality- Adjusted  Years  of  Life  due  to  Different  Diseases”.  National  Bureau  of 
Economic  Research  Summer  Institute,  Boston,  MA,  July  1999. 

Do  People  Consider  Financial  Effects  in  Answering  Quality  of  Life  Questions?”  Plenary 
Abstract,  Society  for  Medical  Decision  Making  Annual  Meeting  Reno  NV 
October  1999. 

“Do  QALYs  Measure  What  We  Want?”  Harvard  School  of  Public  Health,  November 


"Can  Medical  Cost-Effectiveness  Analysis  Identify  the  Value  of  Research?"  Conference  on 
the  Economic  Value  of  Medical  Research  organized  by  the  Lasker  Foundation 
Washington,  DC,  December  1999. 

Theoretical  Foundations  of  Medical  Cost-Effectiveness  Analysis”,  Northeastern  Illinois 
Chapter  of  the  American  Statistical  Association,  Chicago,  IL,  March  2000. 

“Accounting  for  Future  Costs  in  Cost-Effectiveness  of  Prostate  Cancer  Screening  and 

Treatment”.  Robert  Wood  Johnson  Generalist  Physician  Faculty  Scholars  Program 
Annual  Meeting,  December  2000. 

“The  Value  of  Information  in  Cost-Effectiveness  Analysis,  Application  to  Prostate  Cancer 
Screening  and  Treatment"  International  Health  Economics  Association,  (/HEA)  Annual 
Meeting, 

York,  England,  July  2001. 

“Structure  and  Early  Results  of  a  Prostate  Cancer  Decision  Model.”  Society  of  Medical 
Decision  Making  23r  Annual  Meeting.  San  Diego,  CA  October  2001. 

Conclusions 

We  have  conducted  extensive  literature  review  on  the  natural  history,  screening 
treatment,  and  cost-effectiveness  of  prostate  cancer.  To  remedy  the  flaws  in  most  existing 
models,  we  have  developed  the  most  complicated  decision  analytic  model  of  prostate 
cancer  to  date.  In  particular  our  model  is  an  improvement  on  other  models  in  that  it 
accounts  for: 

(1)  Tumor  heterogeneity  and  progression  rates  by  grade. 
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stage^3^385^^011  °f  tUm0r  stage  due  to  the  discrepancy  between  clinical  and  pathologic 

(3)  The  ability  of  screening  and  treatment  to  affect  the  prevalence  of  prostate  cancer. 

(4)  The  effect  of  benign  prostatic  hypertrophy  and  other  prostate  symptoms  and  their 
management  on  the  detection  of  prostate  cancer. 

(5)  The  potential  effects  of  screening  and  treatment  on  quality  of  life,  as  measured  by 
quality  adjusted  life  expectancy. 

We  have  estimated  all  parameters  in  the  model  via  literature  review,  meta-analysis 
secondary  data  analysis,  and  mathematical  modeling.  We  use  the  data  from  SEER 
program  to  validate  the  transition  probabilities  across  stage,  grade  and  morality. 

We  have  completed  the  baseline  cost-effectiveness  of  treatment  of  prostate  cancer. 
Our  results  indicate  treatments  offer  increases  in  non-discounted  life  years  especially  at 
younger  age  below  70  over  all  grades.  However,  when  quality  of  life  weights  were  added, 
quality-adjusted  life  expectancy  was  reduced  compared  with  watchful  waiting.  The  results 
are  sensitive  to  the  patient’s  anxiety  over  the  treatment.  Serious  anxiousness  of  the  patient 
can  favor  the  radiation  and  surgery  over  watchful  waiting.  We  will  continue  this  important 
work  and  conduct  the  extensive  sensitivity  analysis  under  R01  grant  from  the  National 
Cancer  Institute. 
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Appendix  2:  The  Model  Optimization  Results  Compared  with  SEER  Data 


1995  Actual  SEER  Incidence  -  Clinical  Stage  1990s  Predicted  Incidence  ~  Clinical  Stage 


Predicted  Prevalence  -  True  StageABCD  1990s  Cancer  Death  Rates 


1995  Actual  SEER  Incidence  -  Clinical  Stage 


1990s  Predicted  Incidence  -  Clinical  Stage 
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Data  on  variations  in  prostate  cancer  treat¬ 
ment  patterns  may  provide  insight  into  the  ef¬ 
fects  of  new  screening  technologies  such  as 
prostate-specific  antigen  or  into  the  acceptance 
of  new  therapeutic  approaches  such  as  brachy- 
therapy,  or  they  may  raise  questions  about  ac¬ 
cess  to  or  appropriateness  of  treatment1-9  Since 
treatment  is  generally  tailored  to  the  patient’s 
age  and  tumor  stage,  data  on  treatment  pat¬ 
terns  are  most  useful  when  treatment  rates  are 
reported  by  age  and  stage. 

Unfortunately,  even  when  treatment  rates 
are  reported  by  stage,  they  are  often  based  on 
the  most  accurate  stage  rather  than  the  clini¬ 
cal  stage  at  the  time  treatment  decisions  were 
made.6,9,10  The  most  accurate  stage  is  deter¬ 
mined  by  starting  with  the  clinical  stage  and  re¬ 
placing  it  with  the  pathologic  stage  if  surgery 
is  performed  Thus,  assessments  of  tumor  stage 
based  on  clinical  staging  tests,  such  as  digital 
rectal  examination,  are  superseded  by  patho¬ 
logic  information  obtained  from  surgery.  Since 
pathologic  staging  of  prostate  cancer  following 
prostatectomy  often  results  in  a  different  stage 
than  was  determined  by  clinical  staging,11  re¬ 
ports  of  cancer  treatment  based  on  most  accu¬ 
rate  stage  may  not  represent  how  prostate  can¬ 
cer  is  actually  treated  upon  its  clinical 
presentation. 

This  study  used  1997  Surveillance,  Epi¬ 
demiology,  and  End  Results  (SEER)12  data  to 
calculate  treatment  rates  by  age  according  to 
clinical  stage,  and  it  contrasted  these  rates  with 
estimates  of  treatment  rates  according  to  most 
accurate  stage.  The  SEER  program,  which  col¬ 
lects  data  on  cancer  incidence,  treatment,  and 
mortality  from  cancer  registries  that  now  cover 
approximately  14%  of  the  US  population,  is 
believed  to  be  reasonably  representative  of  the 
United  States.13 

Methods 

We  defmed  clinical  stage  as  follows:  A, 
clinically  localized  and  nonpalpable  on  rectal 
examination;  B,  clinically  localized  but  palpa¬ 
ble;  C,  palpable  with  clinical  evidence  of  local 
extension  beyond  the  prostate;  D,  lymph  node 
involvement  or  distant  metastases.  SEER  also 
reports  the  most  accurate  stage  of  cancer,  in¬ 
corporating  any  information  available  from 
prostatectomy.  In  these  cases,  localized  disease 
without  extension  is  classified  automatically 
as  stage  B,  because  staging  guidelines  used  by 
SEER  categorize  all  organ-confined  tumors  as 


stage  B.14  In  addition,  we  classified  patients 
with  lymph  nodes  that  tested  positive  for  can¬ 
cer  as  being  in  pathologic  stage  D. 

We  categorized  treatment  as  surgery 
(prostatectomy),  radiation  (including  brachy- 
therapy),  combined  surgery  and  radiation,  or 
neither  treatment  We  calculated  treatment  rates 
by  age  on  the  basis  of  both  most  accurate  and 
clinical  stage. 

The  sample  comprises  22578  men  older 
than  50  years  with  prostate  cancer.  Of  these, 
131  were  excluded  because  they  had  cysto- 
prostatectomies  and  4865  were  excluded  owing 
to  missing  or  inconsistent  data.  Those  excluded 
did  not  differ  substantially  from  those  included 
by  age,  race,  or  pathologic  stage  where  infor¬ 
mation  was  available  for  comparison. 

Results 

Treatment  Rates 

Overall,  30%  of  patients  receive  surgery, 
32%  receive  radiation,  1%  receive  combined 
therapy,  and  37%  receive  neither  surgery  nor  ra¬ 
diation.  When  patients  are  combined  across  all 
ages  and  staging  is  defmed  by  most  accurate 
stage,  no  stage  A  patients  receive  prostatecto¬ 
mies  (because  SEER  does  not  define  a  patho¬ 
logic  stage  A),  while  37%  of  stage  B  and  78% 
of  stage  C  patients  receive  prostatectomies. 
When  the  data  are  analyzed  by  clinical  stage, 
surgery  rates  are  slightly  higher  than  30%  for 
stages  A  and  B  and  are  only  6%  for  stage  C 
and  1%  for  stage  D.  Radiation  rates  by  most  ac¬ 
curate  stage  for  stages  A,  B,  C,  and  D  are  41%, 
36%,  19%,  and  16%,  while  rates  by  clinical 
stage  are  27%,  38%,  54%,  and  16%. 

Treatment  rates  vary  substantially  by  age, 
whether  by  most  accurate  stage  (Figure  1)  or 
clinical  stage  (Figure  2),  with  the  fraction  of 
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FIGURE  2— Prostatectomy  and  radiation  treatment  rates  for  prostate  cancer 
according  to  age  and  stage  defined  by  clinical  diagnosis. 


patients  receiving  neither  surgery  nor  radiation 
almost  uniformly  increasing  by  age,  but  espe¬ 
cially  after  75  years.  By  most  accurate  stage, 
71%  of  the  youngest  men  with  stage  B  and 
89%  of  the  youngest  men  with  stage  C  cancer 
have  prostatectomies.  By  clinical  stage,  pros¬ 
tatectomy  rates  are  about  67%  for  the  youngest 
men  with  stage  A  and  B  cancer  but  only  19% 


and  8%  for  the  youngest  men  with  stage  C  and 
D  cancer.  The  overstatement  of  surgery  rates  by 
most  accurate  vs  clinical  stage  is  even  greater 
for  men  aged  55  to  74  years.  Because  using 
most  accurate  stage  understates  the  fraction  of 
men  with  clinical  stage  A  cancer  choosing  sur¬ 
gery  and  overstates  the  fraction  with  clinical 
stage  C  cancer  choosing  surgery,  using  most 


accurate  stage  also  substantially  overstates  the 
proportion  of  men  choosing  radiation  for  clin- 


au igc  irv  auu  unucrsraies  me  pro¬ 

portion  choosing  radiation  for  clinical  stage  C 


Treatment  Patterns  by  Age 

Most  men  younger  than  65  years  with 
stage  A  or  B  cancer  choose  surgery,  but  sur¬ 
gery  rates  fall  rapidly  after  70  years  of  age.  Ra¬ 
diation  rates  rise  with  age  until  peaking  in  die 
70s,  when  they  begin  to  decline.  The  overall 
pattern  by  age  is  similar  for  stage  C  patients,  al¬ 
though  surgical  rates  are  generally  lower  and  ra¬ 
diation  rates  are  higher.  Most  men  with  clini¬ 
cal  stage  D  cancer  receive  neither  surgery  nor 
radiation,  although  surgical  rates  are  higher  if 
stage  D  is  defined  by  most  accurate  stage. 

Pathologic  Restaging 

Because  SEER  does  not  identify  a  patho¬ 
logic  stage  A,  the  cancer  stage  of  all  men  with 
clinical  stage  A  cancer  is  reclassified  after  sur¬ 
gery;  this  includes  one  man  who  was  found 
not  to  have  cancer  after  prostatectomy.  Cancer 
stage  was  reclassified  for  nearly  half  of  the 
men  with  clinical  stage  B  cancer  and  approx¬ 
imately  one  third  of  the  men  with  clinical 
stage  C  cancer.  Seventy-seven  percent  of 
stage  C  cancers  by  most  accurate  stage  were  the 
result  of  restaging  following  prostatectomy. 


Discussion 

Treatment  Patterns  by  Clinical  vs  Most 
Accurate  Stage 

Our  results  confirm  that  substantial  up¬ 
staging  occurs  after  surgery,  causing  treatment 
rates  based  on  most  accurate  stage  to  substan¬ 
tially  underestimate  surgery  rates  for  patients 
with  clinical  A  and  B  disease  while  overesti¬ 
mating  surgery  rates  for  stage  C  disease.  This 
probably  explains  why  previous  analyses  of 
treatment  rates  based  on  most  accurate  stage 
found  substantially  lower  rates  of  surgery  for 
stage  A  cancer  and  higher  rates  of  surgery  for 
stage  C  cancer  than  we  find  here.6  These  find¬ 
ings  demonstrate  how  important  it  is  that  can¬ 
cer  databases  retain  information  on  both  clin¬ 
ical  and  pathologic  states,  so  that  treatment 
rates  can  be  reported  on  the  basis  of  clinical 
stage. 

Treatment  Patterns  by  Age 

In  contrast  to  prior  studies  that  did  not  re¬ 
port  rates  by  combinations  of  age  and  clinical 
stage,  our  results  show  that  surgery  is  by  far 
the  predominant  method  of  treating  stage  A 
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and  B  prostate  cancer  among  younger  men, 
with  radiation  predominant  among  somewhat 
older  men. 

A  striking  finding  is  that  nearly  a  fifth  of 
younger  men  with  clinical  stage  C  cancer  are 
treated  surgically,  despite  the  decreased  likeli¬ 
hood  that  they  will  benefit  from  surgery.  With 
staging  based  on  the  “most  accurate”  infor¬ 
mation,  this  relationship  might  have  been  ex¬ 
plained  by  patients  with  clinically  observed  A 
or  B  disease  who  were  then  found  to  have 
stage  C  disease.  Our  findings  suggest  this  is 
not  die  case,  and  they  intensify  the  need  to  un¬ 
derstand  whether  such  surgery  is  justified. 

Limitations 

SEER  holds  advantages,  including  the 
comprehensiveness  of  its  population-based 
sample,  continuity  over  time,  and  attention  to 
quality  control,  but  it  also  has  limitations.  First, 
SEER  relies  on  medical  and  pathologic  records 
that  can  be  incomplete.15  Additionally,  SEER 
does  not  report  whether  lymph  node  involve¬ 
ment  was  found  before  surgery  or  as  part  of 
pelvic  lymphadenectomy  performed  immedi¬ 
ately  before  a  prostatectomy  that  was  conse¬ 
quently  aborted  This  lack  of  data  may  cause 
the  stage  and  treatment  of  some  patients  to  be 
misclassified,  a  possibility  that  suggests  the 
value  of  collecting  information  on  the  mecha¬ 
nism  and  timing  of  assessments  of  lymph  node 
involvement  □ 
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1.  Introduction 


Despite  some  recent  slowing  in  the  growth  of  health  care  costs  in  the  US,  health  care  costs 

r'mQnoTb^antially  0Ver  the  past  Several  decades  and  like,y t0  continue  rising  (Smith 

c  ,ll  998).  This  appears  to  be  largely  due  to  the  growth  of  new  technology  (Fuchs  1990* 

™e’  1992).  While  improvements  in  health  are  highly  valued  (Cutler  and  Richardson 
997,  Murphy  and  Topel,  1998),  evidence  from  diverse  methodological  perspectives  sug- 

fSS  m  n?1!?  teChn,°l0gies  may  have  me  value  at  ^  margin  (Eddy,  1990;  Brook  et  al„ 
983,  McClellan  et  al.,  1994).  Cost-effectiveness  analysis  and  other  methods  for  medical 
technology  assessment  have  arisen  to  attempt  to  address  this  important  problem 
One  of  the  main  challenges  faced  by  medical  cost-effectiveness  analysis  has  been  the 
question  of  how  to  perform  these  analyses  in  the  presence  of  uncertainty  about  the  benefits 
and  costs  of  medical  interventions.  The  uncertainty  of  primary  interest  in  this  regard  is 
uncertainty  in  population  level  outcomes,  although  uncertainty  in  outcomes  at  the  individual 
eve  may  be  present  simultaneously.  This  uncertainty  in  population  level  outcomes  may 
result  either  from  limited  evidence  from  clinical  trials  or  the  need  to  extrapolate  based  on 
the  results  of  clinical  trials  using  decision  analysis  and  its  associated  uncertainties  in  the 
structure  and  parameters  of  decision  models.  This  uncertainty  concerning  the  benefits  and 
costs  of  medical  interventions  has  motivated  much  interest  in  sensitivity  analysis  within 
medical  cost-effectiveness  analysis. 

Yet  though  there  have  been  many  proposals  about  how  to  address  uncertainty  in  cost- 
effectiveness  analysis,  there  has  been  relatively  little  discussion  of  the  objectives  for  per¬ 
forming  sensitivity  analysis.  Without  a  clear  understanding  of  these  objectives,  it  is  difficult 
to  know  by  what  criterion  to  assess  the  merits  of  the  many  alternative  approaches  to  sen¬ 
sitivity  analysis.  Thus,  the  lack  of  clarity  concerning  the  objectives  for  sensitivity  analysis 
is  an  important  reason  for  the  continuing  ambiguity  about  how  to  address  uncertainty  in 
cost-effectiveness  analysis. 


This  paper  attempts  to  identify  the  objectives  for  sensitivity  analysis  within  cost- 
effectiveness  analysis  and  to  develop  methods  suited  to  reaching  those  objectives  The 
primary  objectives  of  sensitivity  analysis  are  argued  to  be:  (1)  to  help  a  decision  maker 
make  the  best  decision  in  the  presence  of  uncertainty,  (2)  to  identify  the  sources  of  uncer¬ 
tainty  to  guide  decisions  for  individuals  or  subgroups  with  characteristics  that  differ  from 
a  base  case,  and  (3)  to  set  priorities  for  the  collection  of  additional  information.  This  paper 
studies  these  problems  by  examining  the  implications  of  an  expected  utility  maximization 
model  for  the  optimal  choice  of  medical  interventions  when  there  is  uncertainty  about  the 
costs  and  benefits  of  those  interventions.  The  results  indicate  that  if  the  objective  is  to 
maximize  expected  utility  given  available  information  —  as  is  implicit,  for  example  in 
the  maximization  of  quality-adjusted  life  expectancy  —  and  if  financial  risk  is  effectively 
diversified  through  either  public  or  private  insurance,  then  the  optimal  decision  is  deter¬ 
mined  by  the  ratio  of  the  expected  cost  divided  by  the  expected  benefit.  Other  assumptions 
about  preferences  or  insurance  will  yield  other  conclusions  about  how  to  account  for  un¬ 
certainty  (Mullahy,  1997),  but  also  would  require  different  models  for  cost-effectiveness 
in  the  absence  of  uncertainty  at  the  population  level.  These  findings  also  have  implications 
for  sensitivity  analyses  done  for  other  purposes.  If  the  objective  of  sensitivity  analysis  is 
to  guide  decisions  for  subgroups  that  differ  from  the  base  case,  then  the  ratio  of  expected 
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costs  to  expected  benefits  for  that  subgroup  is  the  appropriate  criterion.  If  the  objective  of 
sensitivity  analysis  is  to  set  priorities  for  the  acquisition  of  additional  information,  then 
the  incremental  increase  in  expected  utility  with  additional  information  is  the  appropriate 
measure  of  benefit.  Though  such  ideal  value  of  information  calculations  may  be  difficult 
to  perform,  other  approaches  to  sensitivity  analysis  with  less  stringent  data  requirements 
may  provide  bounds  on  the  value  of  information.  Together,  these  approaches  suggest  a 
theoretically  grounded  approach  by  which  the  tools  of  medical  cost-effectiveness  analysis 
can  be  used  to  help  set  priorities  for  medical  research.  Following  these  approaches,  it  may 
be  possible  to  draw  upon  the  vast  literature  on  the  cost-effectiveness  of  specific  medical 
interventions  (Elixhauser  et  al.,  1998)  to  address  crucial  needs  for  more  systematic  ways 
to  set  priorities  for  medical  research.  After  active  discussion  between  Congress,  the  Ad-  ' 
ministration,  and  the  leadership  of  the  National  Institutes  of  Health  (NIH)  over  the  valiie 
of  and  priorities  for  Federal  funding  of  biomedical  research,  the  need  for  such  systematic 
approaches  to  identify  priorities  for  research  at  the  NIH  was  recently  highlighted  in  a  report 
of  the  Institute  of  Medicine  (IOM,  1998). 

Section  2  discusses  the  objectives  of  sensitivity  analysis.  Section  3  discusses  the  primary 
methods  currently  used  to  perform  sensitivity  analysis.  Section  4  uses  an  expected  utility 
maximization  model  to  derive  methods  for  optimal  decision  making  in  the  context  of  un¬ 
certainty  about  population  outcomes.  Section  5  extends  the  basic  results  of  Section  4  to 
encompass  uncertainty  at  the  individual  level.  Section  6  uses  the  model  to  derive  methods 
for  sensitivity  analysis  to  guide  decisions  for  individuals  or  subgroups  that  differ  from  a  base 
case.  Section  7  applies  these  principles  to  a  stylized  decision  concerning  a  medical  treatment 
of  uncertain  benefit.  Section  8  uses  the  model  to  derive  methods  to  use  sensitivity  analyses 
to  inform  priorities  for  the  collection  of  additional  information  to  guide  decision  making, 
including  approaches  to  bound  value  of  information  calculations  with  limited  information! 
Section  9  applies  these  ideas  to  a  stylized  model  of  the  decision  whether  to  treat  prostate 
cancer  and  discusses  some  challenges  in  implementing  these  approaches  to  set  priorities 
for  research.  Section  10  concludes. 


2.  Objectives  for  sensitivity  analysis 

In  order  to  begin  to  assess  methods  to  account  for  uncertainty  in  cost-effectiveness  analy¬ 
sis,  it  is  essential  to  consider  the  objectives  in  performing  sensitivity  analyses.  Although  not 
all  of  these  objectives  may  be  relevant  in  every  application,  the  objectives  appear  to  fall  into 
three  broad  categories:  (1)  to  help  a  decision  maker  make  the  best  decision  in  the  presence 
of  uncertainty  about  costs  and  effectiveness,  (2)  to  identify  the  sources  of  uncertainty  to 
guide  decisions  for  individuals  or  groups  with  characteristics  that  differ  from  a  base  case 

and  (3)  to  set  priorities  for  the  collection  of  additional  information. 

2.1.  Decision  making  under  uncertainty  about  cost  and  effectiveness 

This  is  probably  the  most  common  reason  that  sensitivity  analysis  is  performed  in  medi¬ 
cal  cost-effectiveness  analysis,  and  arises  because  the  scientific  literature  often  does  not 
provide  precise  information  concerning  effectiveness  or  costs.  For  example,  the  efficacy 
of  an  immunization  or  the  frequency  and  cost  of  complications  may  not  be  known  with 


\  i: 
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confidence.  Nevertheless,  patients  must  decide  whether  they  want  the  immunization  and 
public  and  private  insurers  must  decide  whether  they  will  cover  it.  Thus,  having  a  mechanism 
to  help  guide  decision  making  when  the  costs  and  benefits  of  a  medical  intervention  are 
uncertain  is  important. 

2.2.  Decision  making  for  individuals  or  subgroups  that  differ  from  a  base  case 

Though  not  frequently  stated  as  a  motivation  for  sensitivity  analysis,  developing  insight 
i  nlo  decisions  faced  by  individuals  or  subgroups  is  also  a  common  motivation  for  performing 
sensitivity  analysis  in  medical  cost-effectiveness  analysis.  For  example,  a  cost-effectiveness 
analysis  for  immunization  of  a  population  would  likely  consider  the  average  risk  of  ac¬ 
quiring  an  infection  in  the  absence  of  immunization.  However,  an  analyst  examining  the 
cost-effectiveness  of  immunization  for  an  individual  or  group  with  a  known  risk  factor  for 
acquiring  some  infection  would  want  to  reflect  that  higher-than-average  risk. 

2.3.  Priority-setting  for  the  collection  of  additional  information 

When  the  conclusions  of  a  cost-effectiveness  analysis  are  altered  by  parameter  values 
that  cannot  be  ruled  out  based  on  the  literature,  the  collection  of  additional  information 
concerning  those  parameters  may  be  justified.  Though  in  practice  it  is  not  frequently  done, 
sensitivity  analysis  can  be  used  to  identify  parameters  that  may  change  the  results  of  a 
decision  analysis  and  those  parameters  may  then  be  studied  more  intensively.  A  few  studies 
have  used  this  approach  to  determine  the  value  of  sample  size  for  clinical  trials  (Claxton 
and  Posnett,  1996;  Homberger,  1998),  or  to  perform  sensitivity  analysis  in  a  decision  model 
by  calculating  the  expected  value  of  perfect  information  concerning  specific  parameters  of 
the  model  (Felli  and  Hazen,  1998). 

Although  these  three  motivations  for  performing  sensitivity  analysis  are  clearly  distinct, 
papers  in  the  literature  commonly  do  not  distinguish  among  them  in  their  discussion  of 
the  sensitivity  analysis.  This  is  important  because  different  methods  for  sensitivity  analysis 
may  be  better  suited  to  different  objectives.  This  is  discussed  further  below. 


3.  Methods  for  sensitivity  analysis 

Before  attempting  to  derive  methods  for  performing  sensitivity  analysis,  it  is  useful  to 
discuss  the  existing  methods.  The  oldest  and  most  commonly  used  forms  of  sensitivity 
analysis  are  univariate  sensitivity  analyses.  Following  these  approaches,  analysts  begin 
with  the  mean  or  modal  values  of  all  the  probabilities  in  their  analysis  and  use  those  to 
calculate  the  costs  and  benefits  for  a  “base  case”  analysis.  The  parameters  are  then  varied 
individually  across  a  range  of  possible  outcomes  to  see  how  the  cost-effectiveness  of  an 
intervention  changes.  In  some  instances,  the  parameter  values  are  varied  over  the  range  of 
all  possible  values,  while  in  other  cases  they  are  varied  across  confidence  intervals  that  are 
drawn  from  the  medical  literature. 

A  major  advantage  of  one-way  sensitivity  analyses  is  that  they  permit  the  analyst  to 
identify  the  effects  of  individual  parameters  on  the  analysis.  Another  advantage  of  one-way 
sensitivity  analyses  is  that  the  results  can  be  easily  calculated  and  reported.  However,  there 
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0  fin  Th<»  k»  7  !  L  L  H'Sh  and  ,ow  estimates  in  the  literature  mieht  be  0  Q8 

of  0.89-0  96  WidchTlteT*1''  *  prol“lio"  ra“  of  0  92  wilh  >  55%  confidence  interval 
chooseTbre^Zra Le“f  Zfc  ”,  °f  pa«2  If  we 

sufficient  precision  to '  “  Pl"  d°W"  ““  ««*  «»»««•  with 

confidence  interval  I  S  ZSf  lheklnKn'e,,“0"  “  worthwhile.  If  we  use  a  95% 

harm  that  could  occur  if  the  true  value  of  theoZ  'i'TiP’ lhe. poteraial  foran  immense 

be  recognized.  Even ifwefindthauhe f  'ha' w0“,d  <“>  “ 
upper  end  of  the  95%  confidence  mrnrvTv  d“lf0',cl'a"8M  for  a  parameter  value  at  the 
we  should  make.  If  the  welfare  benefits  nl  °  ^  •  °W  ^  shouId  chanSe  the  decision 

welfare  loss  at  an  extreme  ofthe  confide,  maJ°"ly  of  ,he  ‘"“™'  ™  large,  and  any 

result  at  the  extreme  should  have  much  influZ™nfce SalTde'Mol'd'T'1''' 

-^u^rrrsav:ur whi  ? 

simultaneously  across  olausihle  or  uLi  V  analyses  in  which  parameters  are  varied 
cems  described  aZe  Sncemina  oS,  ^  “  a"  *«  con- 

problems.  One  problem  with  these  aDoroLZ"''.^  ana,yses’ as  wel[  a'*  some  additional 
that  must  be  performed  rises  exnoneitLi  v i/t  ^  *C  number  of  sensitivity  analyses 

Iem  is  that  assumptions  about  one  parameterTften  have  ^phclfonsTor'8’  An°ther  ^ 
other  parameters.  For  examole  asdimnfimnc  ,k  ,  .  mpllcatl0ns  for  assumptions  about 

may  have  implications  for  the’histoi/of  diseasTundtrT^'  hlSto^of  ^ untreated  disease 
forts  to  examine  the  joint  distribStf "*?«  * 
with  a  similar  population-based  ?3mniin«  ,  a  modeL  11115  approach,  along 

cost-effectiveness  ratios^sometimertenrredstochasti^Zt-effecfiveness^r”611635  and 

“T”  ”  «  -  i»4  o“d  eu,S 

decision  in  the  prcenZfZerKmr"  bec.iuffi,'i0  ,i°‘  queslio"  of  ,he  optimal 
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nf  !TVeneSS  faui0S  W°Uld  n0t  generally  be  meaningful  (Stinnett  and  Paltiel,  1997) 
%theSe  iSSUCS  “ t0  reformuIate  cost-effectiveness  analyses  in  terns 

cxnress^d^n^t^6116^18  MuUahy’  1998)’ in  which  both  costs  and  benefit.  are 

cxpiussed  in  the  common  denominator  of  years  of  life  saved.  While  free  of  some  of  the 

ShSSr00'"?  ’?  “tin,ati"6  rata,  the  utility  of  the  Net 

a  h  Benefit  approach  is  diminished  by  the  fact  it  does  not  allow  easy  comparisons  with 
results  from  traditional  cost-effectiveness  analyses  that  rely  on  cost-effectiveTess  mti^ 
and  is  dependent  on  assumptions  about  the  valuation  of  improvements  in  health  A  related 
approach  with  similar  concerns  is  to  convert  health  benefits  into  a  monetary  value  as  is 
done  in  cost-benefit  analysis  (Tambour  et  al.,  1998). 

,nmpaSSeS?g  meith°dS’ U  is  interesting  to  note  that  while  all  of  them  appear  to  have 

Z  :S^Car  I  X  0bjeCtiVeS  described  above-  none  of  them  are  explicitly  linked 
o  those  objectives.  As  discussed  above,  this  lack  of  clarity  concerning  the  objectives  for 

.  nsitivity  analysis  is  an  important  reason  for  the  continuing  ambiguity  concerning  methods 
: in! mcdica1' 

an  expected  utility  maximization  model  to  attempt  to  develop  an  approach  to  assess  the 
importance  of  uncertainty  about  parameter  values  in  order  to  make  an  optimal  dSsion 

to?ddUnTntyK  ^  SCCti0nS  ^  f0ll°W  ^  examine  adaPt*ion  of  that  apprZ 
to  address  the  other  two  common  objectives  of  sensitivity  analysis  -  the  determination 

collectionof  addySS  *?•  1JdlVlduds  °r  subSrouPs  the  identification  of  areas  where  the 
collection  of  additional  information  would  be  of  value. 

4.  A  deterministic  model  of  health  outcomes  with  uncertainty  about  effectiveness 

wi!h  ndf  !ZlefSe’  Ze  aSSUme  that  there  iS  uncertainty  about  *e  effectiveness  (9  e  0, 

per  vearl  but  thalT  ?  W  (f°r  6Xample’  blood  Pressure  cbecks 

P  y  ).  that  the  outcome  of  that  medical  care  given  9  is  certain.  By  making  this 

abStf3Ct  fr0m;h6  probIem  * ' uncertainty  in  outcome  for  an  ind^uaf,  2 

instead  on  uncertainty  for  a  “representative  consumer”  assumed  to  be  identical  to  all 

other  individuals,  so  that  there  is  no  heterogeneity  in  the  population.  We  return  to  fte^ 

issues  of  individual  level  uncertainty  and  heterogeneity  in  Sections  5  and  6,  however 

anTntenf,  T  ^  P°™hlhtJ  *at  effectiveness  may  affect  both,  the  costs  and  benefits  of 
an  intervention,  we  allow  both  utility  (U)  and  the  costs  of  the  medical  care  (c)  to  depend 

a^k  miXt  be°fC  ~  C(m’  f  )'.]lus  allows  1116  cost  of*  ^  of  medical  care  to  be  uncertain, 
it  might  be,  for  example,  if  it  is  not  known  how  much  those  blood  pressure  checks  and 

resulting  treatments  would  cost.  In  addition,  utility  is  assumed  to  depend  on  nonXedS 

XXrdenote It  rX031  eX,fnditUre’  S°U  =  U{m'  6'x^-  Here  *  is  written  as 

X  9)  1x  9  I  ??  *  "h  ^  IV  f°r  m  t0  satisfy  the  budSet  constraint 
c{m,  9)  +  x(9)  -1  =  0  for  each  level  of  effectiveness.  To  model  cost-effectiveness  we 

assumethat  people  maximize  expected  utility 1  and  take  the  example  of  a  representative 

While  individual  preferences  may  in  fact  be  inconsistent  with  expected  utility  maximization  OALYs  imnliHHv 
ZeforX  ^°Ple  rTimiZe,eTC‘ed  UtiHty-  *  relaxing  «“*  ^sumption  might  be  desirabk.  doing  so  w2 

therefore  involve  a  substantial  reformulation  of  the  way  in  which  health  a  •  u  l 

Of  uncertainty.  This  is  discussed  further  in  Section  10  eVe"  “  'he  abSCnCe 
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eonsumerwho  maximizes  expected  utility  subject  to  budget  constraint  conditional  on  each 
level  of  effectiveness: 

max  J  p(6)U(m,  6,x(6))  dO  such  that  c(m,  6)  +  x(6)  -  I  =  0  for  all  d.  (1) 

Rewriting  this  as  a  Lagrange  multiplier  problem  with  X{9)  as  the  multiplier  for  the  budget 
constramt  at  each  level  of  6,  and  multiplying  each  k(6)  by  p(B)  without  loss  of  generality 

/S  /  p(Wm’ e •  *(®))  d9+j  mpmu  -  c(m,  e)  -  xm  de.  (2) 

This  generates  a  first-order  condition  for  medical  expenditure  which  is 


a U(m,e,x(e » 


de+  f  —de  =  0. 

J  •  dm 


This  implies  that  investment  in  a  medical  intervention  should  occur  to  the  point  at  which 
its  expected  marginal  benefit  (utility)  equals  the  expected  value  of  the  marginal-utility-of- 
mcome-weighted  marginal  cost.  Allowing  the  marginal  utility  of  income  to  depend  on  6 
reflects  the  possibility  that,  either  because  of  changes  in  the  utility  function  or  costs  with  6 
income  might  have  a  greater  or  lesser  marginal  utility. 

For  an  individual,  these  effects  of  uncertainty  about  the  costs  and  effectiveness  of  medi¬ 
cal  interventions  on  the  marginal  utility  of  income  are  clearly  plausible  and  potentially 
important.  If  someone  has  hip  replacement  for  arthritis  at  age  55  and  then  suffers  a  severe 
complication, is  forced  into  early  retirement,  and  requires  around-the-clock  care,  both  their 

Si  t  n,T  Wi"  bC  dirCCtly  aff6Cted  and  their  maroinal  utility  of  income  could 
change  substantially.  In  a  population,  however,  such  effects  are  far  less  compelling  because 

insurance  can  equate  the  marginal  utility  of  income  across  health  states  unless  an  interven- 
lon  eads  to  an  extraordinarily  large  change  in  either  population  health  or  costs.  Thinking 

TV/?!1  I™  Pe,r,SpeCdVe  in  Which  most  extremely  ^pensive  medical  interventions 
ffect  a  relatively  small  number  of  persons  and  most  common  medical  interventions  are  rel- 

w!nCiL  k"  C0.t',t  iS  much  ,ess  likely  that  the  testate)  marginal  utility  of  income 
will  change  substantially  with  uncertainty  about  the  costs  or  benefits  of  a  single  inteiwen- 

Ixn  H-t  thlS  1S  ^  CaSC’  111611  hmX(0)  ^  1  and  the  first-order  condition  for  medical 
expenditures  converges  to 


J  pm 


dU{m,9,x{6)) 


-J  Xp{6) 


3 c(m,  6) 


d9  =  0, 


which  implies  that  the  cost-effectiveness  ratio  is 
/p(^)(3c(m,  g)/3 m)  dd  1 

/ p(9)(dU (m, x(6))/dm) d 6  ~  X'  (5) 

2  Note  that  even  if  changes  in  health  status  led  to  substantia!  changes  in  income  or  the  need  for  non-medical 
^mpin^Tf  T me  C°nStantaCr0SS  individua,s  in  Afferent  health  states,  optimal  insurance  could  still  eqt 
is  nevertheless1  lty  °^1j'C0?le  a^ross  sta(£5-  !n  practice,  of  course,  insurance  will  often  fall  short  of  this  idea],  butthis 
.  ertheless  a  useful  point  of  reference.  Departures  from  perfect  insurance  are  discussed  further  in  Section  10. 
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u  s.  expected  utility  maximization  implies  that  the  optimum  cost-effectiveness  ratio  of 
.  m  uvem.on  m  a  population  under  uncertainty  is  closely  approximated  by  the  ratio  of 

r  C°;  h  L°  e?eCted  benefitS- NOte  that  •“»  “rati0  of  means”  «*»*»  is  analogous  to 
u  uggested  by  Stinnett  and  Paltiel  (1997)  as  the  solution  to  a  constrained  optinfization 

1  .  .  ,n  a  lmear  Programming  context  and  by  Claxton  (1999)  in  a  Bayesian  discrete 

dunce  decision  theoretic  context.  However,  neither  analysis  derives  the  result  directly  from 

"°r  addres“s  possibk  de|>endenc),  °f 

Wlule  this  argument  about  the  dependence  of  the  marginal  utility  of  income  on  9  has  not 

uo^d  thnreihPreV10U  y  m  the  COntext  of  med5cal  cost-effectiveness  analysis,  it  should  be 
oled  that  the  argument  is  quite  similar  to  that  made  by  Arrow  and  Lind  (1970)  concerning 

the  evaluation  of  nsk  m  public  investment  decisions.  There  the  authors  argue  that  the  large 

‘."'  X?.  hC  PUfb  lC  SeCt°r  all0WS  11  to  effectiveIy  eliminate  any  welfare  loss  associated  with 
kme.ss  of  ‘"vestments  by  spreading  the  risk  across  a  sufficiently  large  population  The 
argument  here  relies  both  on  this  diversification  effect  and  the  relatively  modest  magnitude 
almost  any  one  public  health  care  decision  in  the  context  of  overall  health  and  health 


5.  A  stochastic  population  model  with  individual-level  uncertainty  about  outcomes 

UnUke  in  the  deterministic  model  presented  above,  medical  interventions  almost  always 
‘n  °utcomes  f°r  individuals  even  when  there  is  no  population-level  heterogene- 
ty  so  that  all  individuals  share  a  common  set  of  parameters  (0).  Thus,  for  a  set  of  individuals 
indexed  by  j  e  J  who  might  each  experience  health  outcome  ej  e  E,  the  probability  of 

written  as1"8  °UtC°me  *'  ^  *  *  &  be  ^  aS^>  and  ^pected  utility  can  be 
j  pm  II  f  (£j  1 0)Uj(m,  Sj ,  Xj  (e y ,  6))  de y  dy  d 9  such  that 

Cj(m,  £j,  6)  +  xj (ey ,  0)  -  /  =  0  for  all  9,  j,  ej.  (6) 

Following  the  lines  of  the  argument  above,  we  can  construct  state-specific  Lagrange  mul¬ 
tipliers  *y(ey,  0)  and  note  that  if  there  is  (1)  a  large  population  so  that  aggregate  risk  given 
is  negligible,  (2)  full  insurance,  and  (3)  uncertainty  in  the  effectiveness  of  the  interven- 

above?Sth^  2  C°nSeqUences  in  1116  sense  that  ^  d°es  n°t  have  much  effect  on  A.  as  described 


lim  Xj(e,  9)  ->  A.(e,  9)  X  for  all  e  =  (elt 
£y  €  E  and  for  all  9. 


£y,  •  • £/},  where 


(7) 


Thus,  cost-effectiveness  can  be  identified  by  the  ratio  of  expected  costs  to  benefits 
the  presence  of  uncertainty  at  the  individual  level. 


even  in 
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Legend:  Cost  ($)  /  Effectiveness  (Life  Years) 

Fig.  1 .  Simplified  decision  concerning  a  treatment  of  uncertain  benefit  (cost  (US$)/effectiveness  (life  years)). 

6.  Sensitivity  analysis  to  guide  individual  or  subgroup  decisions 

oroWprn  “  d0ne  t0  guide  decisions  for  individuals  or  subgroups,  the 

ThasT  a'ff  Sen,ayKKe,Same  “  f°r  *e  t0tal  P°Pulation.  «cept  that  the  parameter  vector 
0  has  a  different  probability  distribution  p'(9)  than  in  the  overall  population.  This  may 

ccur  i  parameters  for  those  individuals  or  subgroups  are  thought  to  differ  from  those  for 

e  population  as  a  whole.  This  is  the  type  of  heterogeneity  that  most  frequently  motivates 

be  rPb?nayr  in  cost-effectiveness  analysis.  However,  subgroup  analysis  may  also 

dlt  t  6  CS  °f  ^  parameters  for  a  ^bgroup  are  not  known  to  differ  from 

b^  :*TPUlrn  Wh0le’  bUt  **  subP°Pulation  is  more  or  less  well  studied.  In 
both  cases,  the  analysis  differs  only  in  the  probability  distribution  for  the  parameters  with 

cases  in  which  some  parameters  for  subgroups  are  known  with  certainty  addressed  by  a 
simplification  of  the  analysis  in  which  the  marginal  density  for  the  known  parameters  is 

nmhlTT  ?US!  th,erC  IS  n0  UnCert3inty  about  them'3  Accordingly,  the  solution  to  this 
problem  for  individuals  or  subgroups  is  again  the  ratio  of  the  expected  value  of  costs  to  the 


7.  Application  to  a  stylized  decision  concerning  a  treatment  of  uncertain  benefit 

Fig.  1  describes  a  stylized  decision  concerning  an  intervention  of  uncertain  benefit  For 
simplicity,  the  intervention  is  assumed  to  cost  US$  10,000  with  certainty.  Uncertainty  is 
assumed  to  exist  only  with  respect  to  benefits;  it  is  assumed  that  there  is  a  90%  chance 

pllme'teSand  nZT  ^  ^  ^  ^  8nd  ^  Now  “*“«  this  heterogeneity  can  be  fully 

parameterized  and  partitioned  into  a  certain  part  (Ac)  and  an  uncertain  part  (flu)  so  that  these  pdfs  can  be  fntlv 
parameterized  as  pk($)  =  nfflA-  oA\a  j  _B/m  _  aB\  T  >«  ■  V  n  1  inese  P0 Is  can  be  fuI1y 

to  \  pen  iv  a  P  (  )  p(u>  c  )• In  11115  case,  the  differences  in  the  certain  parameters 

eu).  Thus,  this  framework  incorporates  observable  heterogeneity  as  a  special  case.  g 
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I  wo  exceptions  to  this  are  Claxton  and  Posnett  (1996)  and  Homberger  (1998),  which  focus 
on  the  determination  of  optimal  sample  size  for  a  clinical  trial  from  a  cost-effectiveness 
perceptive  in  a  full  Bayesian  context. 

Adopting  the  expected  utility  approach,  assume  that  for  any  information  set  describing 
the  parameter  distribution,  p(9),  there  is  an  optimal  choice  of  m  as  described  above.  Call 
this  m*(p(9)).  This  implies  an  expected  utility  with  existing  information  (EUo)  of 

J  p{0)U(m*(p(6)),e,x(d))  dd.  (8) 

Now  imagine  that  we  are  able  to  acquire  additional  information  about  6.  Assume  further 
that  the  cost  of  this  research  is  cr.  Though  the  analysis  is  easily  generalized  to  permit  an 
infinite  number  of  possible  outcomes  of  the  experiment,4  assume  for  simplicity  that  there 
are  only  two  possible  outcomes  of  this  experiment:  with  probability  q  that  the  distribution 
°f  6  Is  found  t0  bep'(0)  and  with  probability  (1  -  q)  that  it  is  found  to  be p"{9),  where,  for 
consistency  with  the  initial  prior  distribution,  q*p'(6)  +  (1  -q)*p"{9)  =  p(9).  In  these 
cases,  the  optimal  level  of  medical  expenditure  will  be  m*(p'(0))  and  m*{p"(Q))  and  the 
expected  level  of  utility  is 

q  J  p\e)U(m*(p\6)),  6,  x*\6 ))  d9  +  (1  -  q) 

I  p’\B)U{m\p"m,e,x*\e))  dfl,  (9) 

where  x*  (0)  and  x*"(8)  are  determined  from  the  budget  constraint  net  of  research  costs 
cr  (i.e.  c(m,  6)  +  x(0)  +  cr  -  /  =  0  for  all  9).  It  follows  that  the  change  in  expected  utility 
with  the  collection  of  information,  or  expected  value  of  information  (EVI)  is 

q  J  p'(e)U(m*<P'(9)),9,x*\9))d9  +  a-q) 

J  p'\e)U(m*(P" (9)),  9,  x*" (9))  d$- EU0.  (10) 


In  the  general  case,  we  wish  to  compare  the  expected  utility  resulting  from  the  optimal  decision  m*  given  the 
original  budget  constraint  in  the  absence  of  information  to  the  expected  utility  resulting  from  the  optimal  decision 
in  the  presence  of  the  new  information  subject  to  a  budget  constraint  that  includes  the  cost  of  collecting  information 
(cr).  Thus,  we  compare  the  expected  utility  resulting  from  the  solution  to 

»a«)/  p(0)C/(m'e'-r(0)>d<J  +  / Hd)p(OKc(m.e)  +  x(0)-I]d$ 

^  'individually  given  by" ^  S°'Uti°nS  *° the  0ptimal  decision  problems  for  the;  possible  outcomes  of  the  experiment 

Jaw/ Pi(e'>U(-m'e'x(9»ie  +  f  mpjmc(m.e)+x(e)  +  cr-l]d9. 

where  /  Pj(9)p(j)dj  =  p(d). 

JjtJ 


1 20 


Table  I 


A  Mcltzer/  Journal  of  Health  Economics  20(2001)  109-129 


lulooiiiiiioii  requirements  for  value  of  information  ca|cuiations 

Measure  of  value  ~  rZZZZ,  u..:.  TT-  . 

information  required 


lixpceled  value 
of  Information 
Expected  value  of 
perfect  infornuuion 

Maximum  value 
of  information 

Maximum  value  of  (disease- 
specific)  research 


Expected  gain  in  wel¬ 
fare  from  research 
Expected  gain  from 
perfectly  informative 
specific  experiment 
Maximum  possible 
gain  from  specific 
experiment 
Maximum  possible 
gain  for  target  disease 


Burden  of 
illness 

Yes 

Yes 

Yes 


Yes 


Priors  for  subject 
of  research 

Yes 

Yes 

Minimal  bounds 


Posteriors  for 
subject  of  research 
Yes 


peifonned.P°S't'Ve  P—> «.  «  «.  then  it  shouid  no,  b 

>"  terms, 

the  parameters  required  for  the  calculation  and  th'1"  0rm^Uon  0,1  l,le  Pnor  probabilities  of 
instances,  prior*  may  be  «tima^  tes.H  ^  S T7  V“y  difficu,t  t0  ob,ai"' »■»= 
intervals  or  other  dam from ' ln T*  °f  “■“*  “> 

be  required.  Still,  i,  is  Sy  fta  “  S  t°  "  '“T"5’  P™"*  dm  coltoio»  may 
to  identify  much  mf^adon  “  Wi"  “»  *  ^ 

to  say  much  about  how  an  experiment  is  likely  to  affi*  ^  qUite  difficult 

parameters.  ^  0  a"ect  Postenor  distributions  of  the 

ful.  Table  1  summarizes  a  number  of  wrh  P°f enor  dlstnbutions  would  be  highly  use- 

In  the  case  where  information  on  nriorc  .a^roac  ®s  ^  informational  requirements, 
value  of  perfect  £££££??%££.% 

optimal  choice  of  mif0  is  known  Sin  ’  .  /  ,  9  EU°’  where  m  (0)  is  the 

tive, s  this  provides  an  upper  bound  on  the  ideal  value  of  infonnationcdculationsd^cribed 


above. 


ta  m  “* *“  ">'«’»  “>  «  optima  implies ,b„  n» 

i/p'mu P.rTOdn+t,-,,  j P.m 

=  J  PWU(m'(p(8)),  8,  X(8))  <18. 


(II) 


expected  value  free  informa^on  U  ^oski^For comolT  ^  *!!*““  °f  ,nformation-  This  '"Mies  that  the 
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pract,cal  point>  however> the  advantage  of  the  EVPI  calculation  is  that  it  does  not 
depend  on  the  posteriors.  Indeed,  this  is  probably  one  reason  why  the  EVPI  approach  has 
been  used  in  the  cost-effectiveness  literature  (e.g.  Felli  and  Hazen,  1998)  6  ? 

Drio  s  AUn8^fpVP!lS  SimplCr  t0udetermine  Aan  EVI-  *  «ill  depends  on  knowledge  of  the 
pnors.  An  alternative  measure  that  did  not  depend  on  this  might  also  be  useful  One  such 

measure  is  the  maximal  value  of  information  (MVI)  over  all  possible  values  of  0  e  © 

Tl  ,Allh?2h  “  ak°  °”ly  bC  ”  “pper  bound  EVPI  "S 

therefore,  EVI,  it  depends  only  on  knowing  the  value  function  conditional  on  0  Although  i 

Xea1a7^ 

that  implied  by  a  threshold  analysis  in  which  the  bounds  are  determined  by  the  extreme 

w  r  ,  PTe“r  (aSS“minS- as  iS  value  fuSstone““ 

’.  'T  “  Parametcrs).  Thus,  applying  the  threshold  technique  based  on  the  full 
ranee  of  possible  values  of  a  parameter  can  be  considered  a  bound  on  the  more  general 

Uke  EVft1  f  .lcula“0"' °"1)'  wi,h  riS-reus  infotntation  requirements  Thu^ 

like  EVPI,  the  threshold  approach  based  on  the  full  range  of  values  a  parameter  might  take 

Whe^  rh°nSldeIedia  meth°d  t0  PlaCC  a"  UPPCr  b°Und  °n  the  more  comPlex  EVI  calculation 
ZtZc !Te  CMCatl0nS  SUggeSt  that  the  MVI  or  EVPI  is  low.  the  full  EVI  calculation  is 
at  1  QS?  y'^°te’ m  C°ntraSt’ th3t  the  C°mm0n  practices  °I  assessing  cost-effectiveness 
at  a  95%  confidence  interval  for  a  parameter  or  calculating  stochastic  cost-effectiveness 
intervals  have  no  clear  theoretical  justification. 

tJb'nklng  more  broadly- if  ©  is  enlarged  to  include  any  conceivable  value  of  0  even  if 

conlhl  'S  n0t  P°Mlb  6  WIth  CUITent  technology’ this  tyPe  of  reasoning  can  be  extended  to 
consider  any  possible  research  on  the  parameter  in  question.  For  example  if  the  probabilitv 

of  cure  with  the  best  current  treatment  for  a  disease  is  known  to  be  between  20  and  40%  with 
certainty  and  the  treatment  is  found  not  to  be  worthwhile  (perhaps  because  of  morbidity^ 
one  could  calculate  whether  treatment  would  be  worthwhile  if  the  cure  rate  were  100% 
his  might  be  called  the  maximum  value  of  research  (MVR),  and,  in  turn  can  be  used 
to  generate  an  upper  bound  on  MVI  that  does  not  require  any  data  at  all  concerning  the 

thafledtei  f  °  7  V ^  MVR  C°"Cept  C0U,d  also  be  exPanded  •»  consider  innovSot 

in  ,hes"“c,"re  °f  ,bc  deci!bn  and  ■“*« 


Neverthplpcc  ;n  or  ■  ^  ,  (p.  100).This  seems  t°  suggest  the  modal  value(s)  of  the  parameterts} 

™  i s?,; “«rrSp,b™  ,o7'r ln  *•  “»  -  «-«•. 

an  arbitrary  (optimal  or  suboptimal)  decision  is  present.  P  d  °f  outcomes  from 
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9.  Application  to  a  stylized  model  of  the  decision  whether  to  treat  prostate  cancer 

In  order  to  illustrate  these  approaches,  this  section  examines  a  simplified  model  of  the 
decision  to  treat  prostate  cancer.  A  highly  stylized  model  is  chosen  to  focus  attention  on  the 
methods  rather  than  the  specific  application.  In  this  simplified  model  (Fig.  2),  the  decision  to 
treat  prostate  cancer  is  viewed  as  a  choice  between  radical  prostatectomy  (surgical  removal 
of  the  prostate)  and  “watchful  waiting”  (no  intervention  unless  the  cancer  is  found  to  spread). 
This  decision  is  represented  by  the  two  decision  nodes  in  the  middle  of  Fig.  2.  In  this 
simplified  model,  radical  prostatectomy  is  assumed  to  be  curative,  so  that  the  patient  lives 
out  a  “normal”  life  of  25  years.  However,  radical  prostatectomy  is  assumed  to  have  a 
5%  mortality  rate.  The  outcome  of  watchful  waiting  depends  on  how  quickly  the  cancer 
progresses.  Many  cancers  will  progress  slowly  enough  that  men  die  of  other  causes  before 
they  die  of  prostate  cancer  and  thus  live  a  normal  life  of  25  years.  Other  men  will  progress 
rapidly  and  are  assumed  to  die  of  prostate  cancer  at  10  years.  For  simplicity,  we  assume  that 
quality  of  life  is  not  a  concern  so  that  outcomes  are  measured  in  life  years,  which  are  the 
same  as  quality-adjusted  life  years.  Radical  prostatectomy  is  assumed  to  cost  US$  10,000 
and  the  basic  future  costs  of  survival  are  assumed  to  be  US$  20,000  per  year.  (See  Meltzer 
( 1 997)  for  a  justification  for  including  costs  of  this  nature.) 

However,  the  natural  history  of  prostate  cancer  is  not  as  well  understood  as  suggested  by 
these  assumptions.  In  fact  there  is  much  uncertainty  even  about  average  rates  of  progression 
to  death  from  prostate  cancer,  i.e.  how  aggressive  the  disease  is  on  average.  This  is  the 
dimension  of  uncertainty  on  which  we  focus  in  this  example.  This  is  captured  in  a  stylized 
way  in  Fig.  2  by  the  upper  and  lower  decision  trees  that  differ  in  the  fraction  of  tumors 
that  are  assumed  to  progress  rapidly  (0.085  in  the  “non-aggressive”  case,  and  0.2  in  the 
“aggressive”  case). 
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Watchful  Waiting 
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Legend:  Cost  ($)  /  Effectiveness  (Life  Years) 

Fig.  2.  Simplified  cost-effectiveness  model  for  screening  for  prostate  cancer  with  uncertainty  about  progression 
rates  (cost  (US$)/effectiveness  (life  years)). 
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Panels  I  and2ofTable  1  show  the  results  of  a  cost-effectiveness  analysis  of  the  treatment 
decision  in  the  non-aggressive  and  aggressive  cases.  In  both  cases,  treatment  provides  a 
benefit,  but  in  the  first  case  it  is  a  small  benefit  with  a  cost  per  QALY  of  US$  420,000  and 
in  the  second  case  it  is  a  much  larger  benefit  with  a  cost  per  QALY  of  only  US$  26,000.  If 
we  assume  for  simplicity  that  the  cutoff  for  cost-effectiveness  is  US$  100,000  per  QALY, 
then  the  optimal  decision  in  the  first  case  would  be  watchful  waiting,  while  in  the  second  it 
would  be  treatment. 

The  left  most  part  of  the  decision  tree  reflects  the  fact  that  we  do  not  know  which  of 
these  possibilities  is  the  case  and  places  some  prior  probabilities  on  the  two  arms  (0.2 
aggressive,  0.8  non-aggressive).  Panel  3  of  Table  1  reports  the  expected  benefits  and  costs 
of  the  screening  decision  with  these  priors.  In  that  case,  the  ratio  of  the  expected  costs 
to  expected  benefits  is  US$  47,000,  which  is  cost-effective  by  the  US$  100,000/QALY 
standard.  This  might  seem  surprising  because  of  the  80%  chance  that  progression  was  not 
aggressive,  and  treatment  is  not  even  close  to  cost-effective  by  the  US$  100,000/QALY 
standard  in  that  case.  The  result  is  driven  by  the  20%  chance  that  the  benefit  could  be  much 
larger,  even  though  that  possibility  is  not  very  likely.  This  points  out  the  potential  for  the  ratio 
of  the  expected  value  approach  to  generate  different  results  than  the  standard  probabilistic 
approaches  based  on  thresholds  for  defining  cost-effectiveness  that  do  not  account  fully  for 
both  the  magnitude  and  likelihood  of  the  potential  benefits. 

We  now  turn  to  the  question  of  whether  the  collection  of  additional  information  would 
be  of  value.  Following  the  approach  described  above,  we  begin  by  calculating  the  maxi¬ 
mum  value  of  information.  This  calculation  can  be  done  in  several  ways  requiring  pro¬ 
gressively  more  information.  To  take  an  extreme  example,  assume  that  we  knew  nothing 
about  the  probability  that  prostate  cancer  is  aggressive,  but  only  the  life  expectancy  of 
patients  with  aggressive  cancers  who  are  treated  or  not  treated,  and  the  price  of  prosta¬ 
tectomy.  In  the  absence  of  knowledge  about  the  probability  that  cancers  would  progress 
rapidly,  there  is  no  clear  guidance  about  whether  watchful  waiting  or  prostatectomy  domi¬ 
nates,  so  we  consider  both  cases  as  reference  cases.  Assume  first  that  no  treatment  is 
the  reference  point.  To  get  an  upper  bound  on  the  value  of  information,  one  could  use 
only  information  on  the  life  expectancy  of  treated  and  untreated  patients  and  assume  that 
all  patients  have  aggressive  cancers.  Specifically,  assuming  that  men  who  have  prostate 
cancer  but  are  not  treated  live  10  years  (QALYs),  while  men  who  are  treated  live  25 
years  (QALYs),  the  value  of  treatment  would  be  15  QALYs  x  US$  100,000/QALY  = 
US$  1.5  million  per  patient.  Alternatively,  we  could  assume  that  that  treatment  is  the 
reference  case,  so  that  the  benefit  of  determining  that  treatment  was  not  cost-effective 
would  be  the  cost  savings  from  avoiding  prostatectomy  (US$  10,000)  and  avoidance  of 
treatment-related  mortality  (0.05  mortality  x  25  QALYs  x  US$  100,000/QALY  =  US$ 
125,000)  net  of  any  benefits  of  treatment,  which  add  to  no  more  than  US$  135,000  per 
patient. 

To  use  these  estimates  of  the  maximum  value  of  information  for  a  patient  to  assess 
whether  investment  in  a  study  to  resolve  the  ambiguity  about  the  aggressiveness  of  prostate 
cancer  would  be  worthwhile,  one  might  multiply  these  numbers  by  the  number  of  men  who 
are  found  to  have  prostate  cancer  annually  (100,000)  and  divide  by  some  real  interest  rate 
(0.03)  to  reflect  the  discounted  value  of  the  value  of  that  information  over  time  to  get  the 
maximum  value  of  information  (MVI):  US$  1.5  million  x  100,000/0.03  =  US$  5  trillion 
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if  the  baseline  strategy  is  watchful  waiting  and  USS  0.135  million  x  100,000/0.03  =  US$ 
450  billion  if  the  baseline  strategy  is  prostatectomy.  These  extremely  large  estimates  of  the 
maximum  value  of  information  suggest  the  potential  for  information  of  immense  value  to 
come  from  knowledge  about  the  efficacy  of  prostate  cancer  treatment,  and  exceed  the  cost 
of  any  conceivable  clinical  trial. 

Of  course  these  MVI  calculations  are  an  upper  bound,  and  a  fair  interpretation  of  these 
findings  is  that  the  MVI  is  simply  not  informative  in  this  case,  despite  its  analytical  simplicity 
and  independence  of  assumptions  about  the  fraction  of  cancers  that  are  aggressive.  This 
suggests  that  it  is  worthwhile  to  pursue  the  expected  value  of  perfect  information  (EVPI) 
approach. 

The  EVPI  approach  is  described  in  panel  4  of  the  table.  The  panel  describes  the  expected 
value  of  three  strategies:  watchful  waiting,  radical  prostatectomy,  and  the  optimal  decision 
with  perfect  knowledge  of  the  average  progression  rate  (EVPI).  The  last  two  columns  report 
the  value  of  the  change  in  QALYs  (assuming  USS  100,000/QALY  for  illustration)  and  the 
net  incremental  benefit  of  the  policy  choice  compared  to  the  strategy  immediately  above  it 
in  Table  2. 

The  first  point  to  note  is  that  if  one  made  policy  based  on  the  most  likely  cost-effectiveness 
ratio  (USS  420,000),  one  would  choose  watchful  waiting,  but  if  one  chose  based  on  the  ratio 
of  the  expected  values,  one  would  choose  radical  prostatectomy,  which  yields  a  net  benefit 
of  USS  19,600  (USS  26,000  -  6400)  per  patient  relative  to  watchful  waiting.  This  is  a 
quantified  measure  of  the  expected  gain  from  the  improvement  in  decision  making  by  using 
the  mean  of  the  expected  values  as  opposed  to  basing  the  decision  on  the  most  likely 
cost-effectiveness  ratio,  as  is  generally  done  in  the  “base  case”  reported  by  most  current 
cost-effectiveness  analyses. 

The  second  point  to  note  is  that  the  expected  value  of  the  gain  versus  watchful  waiting  with 
improved  information  is  even  higher  at  USS  26,000  per  patient.  This  implies  an  additional 
gain  of  USS  6400  per  patient  of  the  improved  information  compared  to  the  best  possible 
decision  with  the  initial  information.  Converting  this  patient  level  estimate  of  the  value 
of  research  into  a  population  level  estimate  as  above  suggests  an  EVPI  of  USS  6400  x 
100,000/0.03  =  USS  21  billion.  As  with  the  MVPI,  this  large  EVPI  suggests  that  the  value 
of  information  about  the  efficacy  of  prostate  cancer  treatment  might  far  exceed  the  cost  of 
almost  any  conceivable  clinical  trial. 

Of  course  this  too  is  an  upper  bound  on  the  expected  value  of  information  from  any 
actual  clinical  trial,  since  any  trial  is  likely  to  provide  less  than  perfect  information.  Panel  5 
examines  one  such  case  in  which  an  experiment  has  two  possible  outcomes:  a  50%  chance 
of  an  outcome  that  suggests  that  the  probability  that  prostate  cancer  is  aggressive  is  0.05 
and  a  50%  chance  of  an  outcome  that  suggests  that  prostate  cancer  is  aggressive  is  0.35. 
(Note  this  preserves  the  prior  that  the  probability  that  prostate  cancer  is  aggressive  is  0.2 
since  0.5  x  0.05  +  0.5  x  0.35  =  0.2.)  The  expected  value  of  outcomes  from  watchful 
waiting  and  radical  prostatectomy  given  these  two  possible  outcomes  of  the  experiment 
are  reported  in  the  upper  and  lower  parts  of  panel  5.  In  the  first  case,  the  optimal  decision 
switches  to  watchful  waiting  as  compared  to  prostatectomy  with  the  initial  information, 
which  yields  a  net  surplus  of  US$  600  per  patient.  In  the  second  case,  prostatectomy  remains 
the  optimal  choice,  so  there  is  no  additional  benefit  to  having  done  the  study.  Thus,  the 
expected  net  benefit  is  0.5  x  US$  600  =  USS  300  per  patient.  A  decision  about  the  study 


Table  2 

Value  of  information  for  cost-effectiveness  of  screening  for  prostate  cancer 
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*  ,'fen5,lCOmp™S  ltS  cost  t0  ^  exPected  value  of  the  information  (EVI):  USS 
j  x  ®0,00i0/0'03  -  us$  1  billion.  Therefore,  the  value  of  this  study  would  be  quite 
large,  although  substantially  less  than  the  upper  bound  suggested  by  the  EVPI. 

In  a  similar  manner,  possible  experiments  concerning  all  other  dimensions  of  the  model 
might  be  examined  to  determine  whether  they  would  be  worthwhile.  In  this  way,  it  might  be 
determined  how  much  could  be  gained  by  improved  sensitivity  and  specificity  of  screening 

tests,  decreased  complications  of  treatment,  improved  risk  stratification  prior  to  treatment 
and  so  on.  * 

Clearly,  this  example  does  not  suggest  that  a  comprehensive  attempt  to  perform  a  precise 
calculation  of  the  type  described  would  generate  results  resembling  these  in  magnitude. 
However,  these  simplified  calculations  do  illustrate  the  types  of  calculations  that  might  be 
used  to  assess  the  value  of  research,  including  more  simple  calculations  such  as  the  EVPI 
at  require  less  information.  The  results  also  suggest,  however,  the  potential  for  some  of 
the  approaches  used  in  the  literature,  such  as  the  threshold  (MVI)  or  EVPI  to  provide  only 
very  crude  upper  bounds  on  the  value  of  information.  Just  how  informative  such  bounds 
may  be  in  practice  will  ultimately  be  determined  only  by  detailed  empirical  analysis  of 
specific  clinical  applications. 


10.  Conclusion 


This  paper  has  examined  the  purposes  for  which  sensitivity  analysis  is  performed  in 
medtcal  cost-effectiveness  analysis  and  the  implications  of  an  expected  utility  maximization 
model  for  the  methods  to  perform  such  analyses.  The  analysis  suggests  specific  approaches 
or  optimal  decision  making  under  uncertainty,  specifying  such  decisions  for  subgroups 
and  assessing  the  value  of  collecting  additional  information. 

At  a  theoretical  level,  there  are  several  limitations  of  this  work.  First,  even  with  cer- 
ab°ut  cos*  and  benefits,  cost-effectiveness  analysis  may  not  maximize  the  welfare 
!  '*  (Meltzer  et  a,‘.  1998),  or  society  (Arrow,  1951;  Meltzer  and  Johannesson, 
1998).  Perhaps  more  important  are  issues  about  how  risk  at  the  individual  level  may  affect 
welfare  (Kahneman  and  Tversky,  1979)  that  are  essentially  ignored  by  the  assumptions  of 
perfect  insurance  and  expected  utility  maximization.  Though  this  is  an  important  limitation 
of  QALYs,  it  is  one  that  needs  to  be  addressed  regardless  of  the  issues  about  aggregate 

SCnSitivity  anaIy$is-  Though  concerns  about  aggregate  financial 
and  health  risk  may  be  less  compelling  in  a  social  context  where  the  aggregate  risks  as¬ 
sociated  with  individual  technologies  are  usually  modest,  the  issue  of  how  risk  should  be 
assessed  in  policy  decisions  deserves  further  consideration  because  other  assumptions  about 
preferences  concerning  nsk  or  about  insurance  would  lead  to  different  conclusions  about 
^T^nthi?d0i  oo^ali1!,SUe|S  in  cost~effectiveness  analysis,  including  sensitivity  analysis 
!!?/’  \"7'  ‘ IndeCd’ WhCn  a  medlcaI  intervention  has  major  financial  implications 
a  are  difficult  to  insure  against,  such  as  lost  earnings,  the  marginal  utility  of  income 
cannot  reasonably  be  considered  constant  and  the  results  above  concerning  the  ratio  of 
means  will  no  longer  hold.  This  suggests  that  it  may  be  useful  to  distinguish  between 
uncertainty  in  insured  and  uninsured  costs  in  assessing  the  implications  of  uncertainty  in 
costs  in  cost-effectiveness  analyses.  Additionally,  it  suggests  that  further  characterization  of 
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optimal  decision  making  when  insurance  is  not  complete  would  be  a  valuable  area  for 
future  work. 

Rather  than  using  expected  utility  to  incorporate  preferences  over  uncertain  outcomes,  it 
might  be  argued  that  it  would  be  preferable  to  report  the  joint  distribution  of  benefits  and 
costs.  Nothing  about  this  analysis  suggests  that  such  data  should  not  be  presented.  However, 
using  such  data  to  make  choices  would  still  require  decisions  about  how  to  incorporate 
risk  into  decision  making.  Unlike  traditional  forms  of  sensitivity  analysis,  the  expected 
value  approach  provides  direct  guidance  about  how  the  optimal  decision  varies  with  the 
assumptions  that  are  made. 

At  an  empirical  level,  there  are  important  challenges  in  developing  meaningful  priors  con¬ 
cerning  the  parameters  of  decision  models  (e.g.  probabilities,  quality  of  life  values,  discount 
rates,  etc.).  As  discussed  above,  this  may  often  require  extensive  review  of  existing  data,  pri¬ 
mary  data  collection,  or  even  analyses  based  on  arbitrary  priors.  It  may  also  be  very  difficult 
to  specify  how  research  may  affect  posteriors.  Whether  it  is  possible  to  adequately  address 
these  challenges  will  be  resolved  only  through  efforts  to  apply  these  ideas  empirically. 

These  approaches  to  assess  the  value  of  research  also  pose  additional  challenges.  These 
include  the  interdependence  of  the  benefits  of  related  research,  the  possibility  that  the  re¬ 
search  might  become  less  (or  more)  valuable  over  time  if  technological  or  demographic 
changes  alter  the  management,  frequency  or  natural  history  of  a  disease,  and  the  unpre¬ 
dictability  of  how  the  results  of  research  (particularly  basic  research)  might  be  useful  in 
areas  outside  the  initial  areas  of  inquiry  (serendipity).  The  difficulty  of  these  issues  implies 
that  the  sort  of  formal  analyses  suggested  here  are  more  likely  to  be  useful  for  evaluating 
clinical  research  than  basic  research. 

Despite  these  theoretical  and  empirical  challenges,  the  importance  of  making  good  de¬ 
cisions  about  the  allocation  of  resources  to  medical  interventions  and  medical  research 
suggest  that  work  in  this  area  be  an  important  priority.  It  is  encouraging  in  this  regard  that 
the  recent  IOM  report  on  improving  priority  setting  at  the  NIH  recommended:  “In  setting 
priorities,  NIH  should  strengthen  its  analysis  and  use  of  health  data,  such  as  burdens  and 
costs  of  diseases,  and  on  data  on  the  impact  of  research  on  the  health  of  the  public”  (IOM 
1998,  p.  11). 

On  the  other  hand,  the  limited  number  of  cases  where  cost-effectiveness  analysis  has 
strongly  influenced  medical  resource  allocation  and  the  likely  resistance  of  medical  re¬ 
searchers  to  having  research  proposals  evaluated  by  formal  criteria  suggest  that  formal 
techniques  to  set  priorities  for  research  will  have  to  prove  their  value.  It  is  possible  that 
cost-effectiveness  analysis  may  enhance  its  influence  if  it  can  address  key  methodological 
challenges  in  measuring  benefits  and  costs,  and  techniques  for  sensitivity  analysis.  There 
may  also  be  less  resistance  to  the  use  of  cost-effectiveness  analysis  in  policy  decisions,  such 
as  allocation  of  research  funds,  than  to  its  use  in  decisions  to  ration  medical  treatments. 
Nevertheless,  formal  techniques  to  inform  priorities  for  research  seems  more  likely  to  gain 
acceptance  through  instances  where  neglected  areas  of  research  can  be  identified  through 
formal  analysis  than  through  instances  where  research  is  suggested  to  be  of  little  value. 
Consistent  with  this,  threats  to  increases  in  the  NIH  budget  due  to  Congressional  questions 
about  the  value  of  increased  appropriations  for  research  and  NIH  priorities  in  allocating 
research  funds  were  an  important  motivation  for  the  IOM  report  that  encouraged  efforts  to 
use  formal  approaches  to  determine  the  value  of  research. 
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Forum:  Recommendations  for 
Cost-Effectiveness  Analysis 

Inconsistencies  in  the  “Societal 
Perspective”  on  Costs  of  the  Panel  on 
Cost-Effectiveness  in  Health  and  Medicine 

DAVID  MELTZER,  MD,  PhD,  MAGNUS  JOHANNESSON,  PhD 


A  key  recommendation  of  the  recent  Pane!  on  Cost-Effectiveness  in  Health  and  Med¬ 
icine  was  that  cost-effectiveness  analyses  be  carried  out  from  a  societal  perspective. 
The  authors  show  that  two  of  the  Panel’s  recommendations  concerning  costs  are  not 
consistent  with  a  societal  perspective,  and  how  to  correct  those  inconsistencies.  In  its 
recommendations  concerning  costs  resulting  from  morbidity,  the  Panel  advises  ex¬ 
cluding  lost  income  from  costs  in  the  belief  that  individuals  take  income  changes  into 
account  when  they  respond  to  the  quality-of-life  questions  that  are  used  to  calculate 
quality-adjusted  life  years  (QALYs).  It  is  shown  that  even  if  individuals  do  consider 
income  changes  in  responding  to  these  quality-of-life  questions,  this  recommendation 
would  seriously  underestimate  production  losses  due  to  morbidity,  since  individuals  do 
not  bear  a  major  part  of  lost  production.  In  its  recommendations  concerning  costs 
resulting  from  mortality,  the  Panel  does  not  require  that  health  care  costs  for  “unre¬ 
lated”  illness  and  non-health  care  consumption  and  production  during  added  life  years 
be  included  in  the  Reference  Case.  It  is  shown  that  omitting  these  costs  will  seriously 
distort  comparisons  of  programs  at  different  ages  and  favor  programs  that  extend  life 
over  those  that  improve  quality  of  life.  This  can  be  corrected  by  including  total  con¬ 
sumption  minus  production  in  added  life-years  among  costs.  Key  words :  cost-effec¬ 
tiveness  analysis;  societal  perspective;  public  policy;  resource  allocation;  Panel  on 
Cost-Effectiveness  in  Health  and  Medicine.  (Med  Decis  Making  1999;19:371-377) 


The  rapid  advancement  of  medical  technology  and 
the  resulting  increase  in  health  care  costs  have  in¬ 
creased  interest  in  the  economic  evaluation  of 
health  care  programs.  There  are  various  methods 
for  the  economic  evaluation  of  medical  interven¬ 
tions,  but  the  one  that  is  currently  most  popular  is 
cost-effectiveness  analysis.  Following  this  approach, 
costs  are  measured  in  monetary  terms  and  effec¬ 
tiveness  is  measured  in  non-monetaiy  terms,  e.g., 
quality-adjusted  life  years  (QALYs).  A  problem  with 
the  method  is  that  it  has  not  been  clear  how  to  de- 
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fine  costs  and  effectiveness.  This  has  led  to  great 
variability  in  what  is  included  in  measures  of  costs 
and  effectiveness,  which  has  caused  problems  in 
comparing  and  interpreting  the  results  of  cost-ef¬ 
fectiveness  analyses.1 

Because  of  these  problems,  the  Panel  on  Cost- 
Effectiveness  in  Health  and  Medicine  was  convened 
in  1993  by  the  U.S.  Public  Health  Service  to  develop 
recommendations  for  the  conduct  of  cost-effective¬ 
ness  analysis.  The  official  report  of  the  Panel2  was 
summarized  in  a  series  of  three  papers  in  JAMA.3'5 
A  key  recommendation  of  the  Panel  was  that  cost- 
effectiveness  analyses  be  carried  out  from  a  societal 
perspective,  including  all  the  costs  and  conse¬ 
quences  of  health  interventions  no  matter  to  whom 
they  accrue.  However,  the  Panel  also  recommended 
that  most  productivity  gains  and  losses  resulting 
from  medical  interventions  that  reduce  morbidity 
(often  referred  to  as  morbidity  costs)  not  be  in¬ 
cluded  among  the  costs  in  cost-effectiveness  anal¬ 
yses.  In  addition,  they  concluded  that  it  was  not  nec¬ 
essary  to  include  certain  classes  of  medical  or 
non-medical  expenditures  in  additional  years  of  life 
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that  result  when  a  medical  intervention  reduces 
mortality  (often  referred  to  as  mortality  costs). 

Here  we  argue  that  these  recommendations  of  the 
Panel  are  not  fully  consistent  with  a  societal  per¬ 
spective  since  they  do  not  correctly  account  for  costs 
resulting  from  the  effects  of  changes  in  morbidity 
on  productivity  and  the  effects  of  mortality  on  pro¬ 
ductivity  and  consumption  in  added  years  of  life. 
This  implies  that  cost-effectiveness  analyses  carried 
out  in  accordance  with  the  recommendations  of  the 
Panel  would  not  lead  to  an  efficient  use  of  resources 
from  a  societal  perspective.  Below  we  show  why  the 
recommendations  of  the  Panel  are  inconsistent  with 
a  societal  perspective,  and  how  the  definition  of 
costs  in  cost-effectiveness  analysis  can  be  made 
consistent  with  a  societal  perspective. 

The  Treatment  of  Morbidity  Costs 

Apart  from  affecting  health  care  costs  (and  other 
direct  costs),  a  health  care  program  may  also  affect 
the  production  of  goods  and  services  by  an  individ¬ 
ual,  whether  in  the  market  or  outside  the  market. 
For  simplicity,  we  can  call  production  inside  and 
production  outside  the  market,  respectively,  “in¬ 
come”  and  “leisure.”  These  effects  of  morbidity  on 
production  are  recognized  by  the  Panel.  However, 
the  Panel  argues  that  changes  in  market  and  non- 
market  production  borne  by  individuals  should  not 
be  included  among  the  costs.2'13  306  Their  justification 
for  this  recommendation  is  based  on  the  idea  that 
people  already  include  these  costs  in  estimating 
quality-of-life  weights  used  to  construct  QALYs,  so 
that  it  would  be  double-counting  to  include  them 
among  the  costs.  The  Panel  also  recognized  that  not 
all  productivity  losses  would  be  borne  by  individuals 
and  concluded  that  “effects  of  lost  productivity 
borne  by  others  (e.g.  employers,  coworkers),  includ¬ 
ing  ‘friction  costs/  when  significant,  should  be  in¬ 
cluded  in  the  numerator.”2,13186 

While  the  general  principle  described  by  the  Panel 
that  effects  not  incorporated  in  the  denominator  be 
included  in  the  numerator  is  theoretically  sound,  it 
is  not  clear  that  their  assumptions  concerning  what 
people  actually  include  when  responding  to  the 
QALY  questions  used  to  construct  the  denominator 
are  correct.  In  particular,  it  is  not  known  whether 
individuals  take  changes  in  income  or  leisure  into 
account  when  they  respond  to  the  quality-of-life 
questions  used  to  construct  QALY  weights.  It  seems 
plausible  that  changes  in  leisure  time  may  be  incor¬ 
porated  in  the  QALY  weights,  since  individuals  may 
weigh  these  effects  as  they  respond  to  questions 
evaluating  quality  of  life  in  specific  health  states. 
However,  the  argument  that  changes  in  market  pro¬ 
duction  are  reflected  in  the  quality  weights  is  not 
credible.  One  instrument  used  to  generate  QALY 
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weights  for  health  states,  the  Health  Utilities  Index, 
explicitly  instructs  individuals  being  interviewed  to 
assume  that  their  financial  circumstances  would  not 
vary  with  health  status.6  All  other  elicitation  pro¬ 
cesses  provide  respondents  with  no  guidance  about 
what  to  assume  about  the  economic  consequences 
of  health  states  when  responding  to  QALY  elicitation 
questions.  Without  guidance,  it  is  possible  that  in¬ 
dividuals  may  or  may  not  incorporate  financial  ef¬ 
fects. 

The  Panel  attempts  to  address  this  concern  by 
recommending  that  “for  the  reference  case,  health- 
related  quality  of  life  should  be  captured  by  an  in¬ 
strument  that,  at  minimum,  implicitly  incorporates 
the  effects  of  morbidity  on  productivity  and  lei¬ 
sure.”2'13306  Thus,  the  Panel’s  recommendations  im¬ 
ply  that  instruments  such  as  the  Health  Utilities  In¬ 
dex  should  not  be  used  for  utility  assessment. 
However,  the  Panel  also  recommends  that,  “when 
instruments  used  to  measure  health  states  are  silent 
concerning  the  consideration  of  lost  income,  we  as¬ 
sume  that  financial  effects  have  been  considered  by 
the  respondent  and  that  it  is  therefore  not  necessary 
to  account  for  these  effects  in  the  numerator.”2'13209'4 
Unfortunately,  there  is  no  evidence  that  this  as¬ 
sumption  is  valid.  To  have  confidence  in  the  Panel’s 
position  that  personal  financial  effects  are  already 
incorporated  in  QALY  weights,  either  respondents 
should  be  explicitly  told  to  include  these  effects  or 
empirical  research  should  be  performed  to  dem¬ 
onstrate  that  this  is  the  case. 

Moreover,  even  if  individuals  do  consider  per¬ 
sonal  financial  consequences  in  responding  to  QALY 
elicitations,  the  recommendations  of  the  Panel  are 
not  consistent  with  a  societal  perspective  because 
they  do  not  adequately  reflect  the  extent  to  which 
individuals  would  undervalue  changes  in  produc¬ 
tion  if  they  were  to  take  only  personal  financial  con¬ 
sequences  into  account  in  responding  to  QALY 
questions.  In  particular,  although  the  Panel  did  rec¬ 
ognize  the  need  to  include  costs  borne  by  others 
such  as  employers  and  coworkers, 2,p  306  they  did  not 
adequately  recognize  the  variety  of  forms  of  insur¬ 
ance  and  taxes  that  prevent  individuals  from  bearing 
the  full  consequences  of  changes  in  their  produc¬ 
tion.  When  there  is  fully  paid  public  or  private  dis¬ 
ability  insurance,  for  instance,  a  person  does  not  ex¬ 
perience  any  change  in  income  when  he  or  she 
experiences  illness,  and  might  therefore  not  con¬ 
sider  effects  on  productivity  when  responding  to 
QALY  questions.  Nevertheless,  the  Panel  advises  ex¬ 
cluding  payments  such  as  these  from  costs  on  the 
grounds  that  they  are  transfer  payments.2'13307  The 
result  is  that  the  real  decrease  in  productivity  re¬ 
sulting  from  the  illness  is  reflected  neither  in  the 
numerator  nor  in  the  denominator,  so  that  the  anal¬ 
ysis  is  inconsistent  with  a  social  perspective.7  Al- 
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though  transfer  payments  do  not  represent  true  so¬ 
cial  costs,  the  accounting  approach  advocated  by  the 
Panel  results  in  the  rather  unintuitive  implication 
that  the  true  social  costs  of  lost  productivity  would 
be  captured  only  by  including  these  transfer  pay¬ 
ments  from  disability  insurance  as  a  cost. 

But  even  in  the  absence  of  public  or  private  dis¬ 
ability  insurance,  the  presence  of  income  and  pay¬ 
roll  taxes  implies  that  an  individual  bears  only  a 
fraction  of  the  consequences  of  changes  in  produc¬ 
tion.  This  is  because  the  true  value  of  the  lost  pro¬ 
duction  of  a  worker  who  is  absent  from  work  is 
equal  to  the  amount  of  money  the  employer  is  will¬ 
ing  to  pay  for  that  work,  which  also  includes  the 
income  tax  paid  by  the  worker  and  the  payroll  tax 
paid  by  the  employer.  It  is  important  to  note  that, 
unlike  productivity  losses  related  to  “friction  costs," 
which  have  been  the  focus  of  several  recent  articles 
discussing  the  appropriate  treatment  of  productivity 
costs,7"9  and  rest  on  debatable  assumptions  about 
the  rigidity  of  labor  markets,  these  costs  last  as  long 
as  the  disability  lasts  and  may  be  veiy  large.  To  il¬ 
lustrate  the  magnitude  of  this  effect,  note  that  with 
a  payroll  tax  of  30%  of  the  wage  rate  and  an  income 
tax  of  30%,  the  individual  will  bear  only  about  50% 
of  the  value  of  the  change  in  production  due  to  dis¬ 
ease  even  if  no  payment  for  sick  leave  exists.  Failing 
to  account  for  these  changes  in  production  will  thus 
seriously  underestimate  the  value  of  gains  or  losses 
in  production,  even  if  individuals  reflect  the  changes 
in  personal  income  in  responding  to  questions  that 
elicit  QALY  weights.  Unfortunately,  the  Panel  never 
discusses  the  question  of  whether  income  and  pay¬ 
roll  taxes  should  be  included  in  costs  except  in  one 
table,  where  the  implication  appears  to  be  that  they 
should  not  be  included.2'*5 187 

The  Panel  also  advised  including  all  health  care 
costs  in  measures  of  costs.  The  problem  with  this 
recommendation  is  that  if  people  incorporate  per¬ 
sonal  financial  consequences  into  QALY  weights, 
they  should  presumably  also  incorporate  out-of- 
pocket  medical  expenditures,  and  including  all 
health  care  costs  in  measures  of  costs  would  lead 
to  double-counting  of  the  out-of-pocket  costs.  In  the 
United  States,  this  could  lead  to  substantial  errors 
in  estimates  of  both  the  absolute  and  the  relative 
cost-effectiveness  of  medical  interventions,  since 
about  20%  of  health  care  expenditures  are  paid  for 
out-of-pocket,  and  the  fractions  paid  out-of-pocket 
vary  a  great  deal  across  different  types  of  medical 
procedures,  depending  on  insurance  coverage.10 

A  similar  problem  arises  with  the  Panel’s  recom¬ 
mendation  to  include  the  value  of  time  lost  from 
work  or  leisure  activities  in  order  to  participate  in  a 
health  intervention  (such  as  an  exercise  program)  in 
measures  of  costs.  If  individuals  incorporate  effects 
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on  personal  economic  status  and  leisure  into  QALY 
weights,  they  should  presumably  already  be  incor¬ 
porating  these  effects  into  their  QALY  weights,  and 
including  them  in  measures  of  costs  as  well  would 
again  lead  to  double-counting  of  these  costs.  This 
may  not  be  much  of  a  problem  when  the  treatment 
is  infrequently  required,  but  could  be  veiy  impor¬ 
tant  when  the  treatment  is  time-consuming  and 
chronic,  as  is  hemodialysis  for  end-stage  renal  dis¬ 
ease. 

Therefore,  even  if  individuals  consider  the  private 
economic  consequences  of  illness  in  responding  to 
QALY  elicitations,  the  recommendations  of  the  panel 
are  not  consistent  with  a  societal  perspective  on 
costs.  Indeed,  it  is  only  under  the  very  specific  as¬ 
sumption  that  individuals  incorporate  into  their 
QALY  weights  exactly  what  the  Panel  has  chosen  to 
exclude  from  costs  that  the  Panel’s  recommenda¬ 
tions  can  be  considered  to  be  consistent  with  a  so¬ 
cietal  perspective  on  costs.  To  be  precise,  this  means 
that  individuals’  QALY  weights  would  have  to  reflect 
the  value  of  lost  leisure  and  the  total  value  of  lost 
productivity  in  the  market,  including  not  only  take- 
home  wages  but  also  taxes  that  the  government 
would  have  received  from  individuals  as  income 
taxes  and  from  their  employers  as  payroll  taxes. 
Moreover,  individuals  would  have  to  incorporate 
these  losses  into  their  QALY  weights  only  if  the  pro¬ 
ductivity  change  was  due  to  illness  and  not  if  it  was 
due  to  participation  in  a  health  care  intervention. 
Also,  despite  reflecting  these  changes  in  their  per¬ 
sonal  welfare  due  to  changes  in  income  and  leisure 
in  their  QALY  weights,  individuals  would  have  to  ex¬ 
clude  changes  in  medical  costs  from  their  QALY 
weights,  even  if  the  costs  were  paid  out  of  pocket. 

Thus,  the  Panel’s  assumption  that,  when  health 
status  measures  are  silent  on  financial  issues,  finan¬ 
cial  effects  have  “been  considered  by  the 
respondent’’4  in  such  a  way  as  to  lead  to  consistent 
definitions  of  benefits  and  costs  seems  extremely 
unlikely  to  be  valid.  Moreover,  even  the  recommen¬ 
dation  that  respondents  be  explicitly  told  to  “con¬ 
sider  the  full  range  of  impacts  of  the  health  status 
change,  including  loss  of  income  and  leisure 
activities’’4  will  not  generate  responses  consistent 
with  a  societal  perspective  on  costs.  Indeed,  the  Re¬ 
port2  recognizes  that  when  people  do  not  take  “ef¬ 
fects  of  lost  productivity  borne  by  others”  into  ac¬ 
count  [p.  306],  those  effects  must  be  included  in 
costs.  Nevertheless,  the  report  does  not  follow 
through  with  that  principle  by  recommending  that 
lost  tax  revenue  be  included  in  costs  or  that  out-of- 
pocket  expenditures  on  health  care  be  excluded 
from  health  care  costs. 

These  inconsistencies  in  the  Panel’s  recommen¬ 
dations  could  be  corrected  by  clarifying  the  process 
of  QALY  elicitation  to  be  explicit  about  what  eco- 
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nomic  consequences  individuals  are  to  consider  and 
then  using  an  appropriate  definition  of  costs.  One 
possibility,  which  is  similar  to  the  recommendations 
of  the  Panel,  would  be  to  encourage  individuals  to 
take  into  account  all  the  personal  financial  conse¬ 
quences  of  changes  in  their  market  and  non-market 
productivity  due  to  illness  (i.e.,  changes  in  after-tax 
income  and  out-of-pocket  medical  costs  and 
changes  in  leisure)  when  responding  to  QALY  elici¬ 
tation  questions.  However,  in  that  case,  the  part  of 
health  care  costs  and  production  change  not  borne 
by  the  individual  should  be  included  in  the  estima¬ 
tion  of  costs.  Alternatively,  individuals  could  be  told 
to  provide  QALY  weights  based  on  the  assumption 
that  their  financial  circumstances  would  not  vary 
with  health  status,  but  that  their  non-market  pro¬ 
ductivity  would  vaiy  with  health  status  (i.e.,  that  they 
have  paid  sick  leave  and  do  not  pay  anything  out  of 
pocket  for  health  care  costs).*  The  full  health  care 
costs  and  the  full  change  in  production  should  then 
be  included  in  the  cost  estimation. 

Either  of  these  approaches  could  be  justified  in 
theory,  but  from  a  practical  viewpoint  the  second 
approach  seems  preferable,  since  the  analysis  be¬ 
comes  much  more  straightforward  to  perform  and 
to  interpret.  It  avoids  collecting  detailed  information 
about  paid  sick  leave  and  out-of-pocket  payments, 
which  would  be  necessary  following  the  first  ap¬ 
proach.  The  advantage  of  the  second  approach  is 
even  more  obvious  if  QALY  weights,  as  recom¬ 
mended  by  the  Panel,  are  elicited  from  the  general 
population  without  experience  of  the  health  states 
to  be  assessed.  This  makes  the  second  approach 
preferable,  because  otherwise  individuals  would 
have  to  make  uninformed  guesses  about  the  in¬ 
comes  in  different  health  states,  or  the  income  level 
would  have  to  be  explicitly  incorporated  as  an  at¬ 
tribute  in  the  health  status  classification  system 
used. 


The  Treatment  of  Mortality  Costs 

The  correct  treatment  of  mortality  costs  is  a  long¬ 
standing  and  active  area  of  controversy  in  medical 
cost-effectiveness  analysis.  As  recognized  by  the 
Panel,  and  further  clarified  subsequent  to  the  pub¬ 
lication  of  its  report,  the  theoretically  correct  way  to 
include  mortality  costs  is  to  include  the  difference 


*It  may  require  some  experimentation  to  determine  exactly 
the  best  approach  to  doing  this,  but  we  would  propose  the  fol¬ 
lowing  sentence  as  one  concrete  possibility  For  example,  in  in¬ 
troducing  a  time-trade-off  question,  we  might  suggest  the  state¬ 
ment:  “In  answering  this  question,  please  assume  that  your 
financial  circumstances  would  not  be  affected  by  the  illness,  as 
might  be  the  case  if  you  did  not  have  to  pay  anything  for  health 
care  and  you  received  disability  payments  that  completely  re¬ 
placed  any  lost  earnings." 
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between  total  consumption  and  total  production 
during  life  years  gained  by  a  medical  intervention  as 
a  cost  in  cost-effectiveness  analysis.11"13  However, 
the  Panel  did  not  recommend  that  these  costs  be 
included  in  all  cases.  In  the  case  of  health  care  costs 
for  'unrelated”  illnesses  during  life  years  gained  by 
an  intervention,  they  concluded  that  cost-effective¬ 
ness  ratios  can  be  estimated  with  or  without  includ¬ 
ing  these  costs,  but  that  a  sensitivity  analysis  should 
be  performed  whenever  those  costs  had  a  signifi¬ 
cant  effect  on  the  results.  In  the  case  of  non-health 
care  costs  during  gained  life  years,  the  summaiy 
recommendations  of  the  Panel  did  not  include  any 
relevant  recommendation,  though  other  parts  of  the 
report  suggested  conditions  under  which  these 
costs  could  be  omitted2,pp  48186  and  a  later  article 
summarizing  its  recommendations  suggests  that 
these  costs  be  excluded.4 

The  Panel  discusses  several  different  arguments 
to  justify  excluding  future  costs  [p.  186].  One  is 
purely  political — namely  that  people  may  not  be 
willing  to  accept  including  such  costs  in  medical 
cost-effectiveness.  We  postpone  discussion  of  this 
argument  until  the  conclusion.  The  other  argu¬ 
ments  are  summarized  as  follows: 

In  addition  to  this  difference  of  opinion,  there  are 
practical  difficulties  in  including  costs  for  unrelated 
illness  in  added  years  of  life.  Existing  data  may  not 
be  adequate  to  capture  future  resource  use  of  all 
unrelated  diseases;  in  addition,  it  may  be  unduly  dif¬ 
ficult  to  ascertain  the  effect  of  an  intervention  on  the 
range  of  future  causes  of  morbidity  and  death.  Fi¬ 
nally,  if  these  costs  are  included,  non-health  care 
costs  in  added  years  of  life  should  also  be  included. 

As  discussed  in  Chapter  2,  the  exclusion  of  non¬ 
health  care  costs  is  acceptable  if  these  costs  add  a 
constant  cost  to  each  year;  however,  no  research  has 
been  done  to  determine  whether  this  is  the  case. 

Because  of  the  practical  concerns  and  unresolved 
theoretical  issues  surrounding  the  inclusion  of  fu¬ 
ture  costs  for  unrelated  illness  in  added  years  of  life, 
our  recommendation  for  the  Reference  Case  is  that 
analysts  use  their  discretion  in  including  or  exclud¬ 
ing  these  costs.  Like  other  costs  and  consequences, 
the  rule  of  reason  applies  to  these  costs:  If  they  are 
small  compared  to  the  magnitude  of  the  C/E  ratio, 
they  can  be  omitted  without  affecting  the  analysis  re¬ 
sults  in  any  case.  If  they  are  large,  we  recommend 
that  the  analyst  conduct  a  sensitivity  analysis  to  as¬ 
sess  their  effect.”  [p.  186] 

To  assess  the  Panel's  position  on  future  costs,  it 
is  useful  to  review  these  arguments  systematically. 
First,  there  is  the  issue  of  the  adequacy  of  data  to 
estimate  future  costs  for  unrelated  diseases.  While 
the  Panel  is  correct  that  this  may  be  difficult  to  do 
precisely  and  is  an  area  where  further  research 
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would  be  valuable^  they  also  describe  a  number  of 
data  sources  with  which  “first”  estimates  could  be 
derived  [pp.  186-8]  and  there  are  now  estimates  in 
the  literature.12,14  In  any  case,  it  seems  difficult  to 
argue  that  including  an  implicit  estimate  of  zero  by 
omitting  unrelated  costs  would  be  preferable  to  an 
imprecise  estimate,  especially  with  appropriate  sen¬ 
sitivity  analysis.  It  is  not  clear  what  the  Panel  means 
by  difficulty  in  ascertaining  “the  effect  of  an  inter¬ 
vention  on  the  range  of  future  causes  of  morbidity 
and  death,”  but  if  it  concerns  related  illnesses,  then 
the  problem  applies  equally  to  costs  for  related  ill¬ 
ness,  whose  inclusion  is  recommended  by  the 
Panel.  If  it  concerns  unrelated  illness,  the  very  def¬ 
inition  of  such  costs  suggests  that  they  should  be 
unaffected  by  the  intervention  and,  again,  it  seems 
unlikely  that  an  estimate  of  zero  would  be  preferable 
to  an  imprecise  estimate  with  appropriate  sensitivity 
analysis. 

The  recommendation  that  whenever  future  costs 
for  unrelated  illness  are  included,  future  non¬ 
health  care  costs  also  be  included  is  consistent  with 
theory,  but  implies  that  it  is  acceptable  not  to  in¬ 
clude  such  non-health  care  costs  so  long  as  costs 
of  unrelated  illnesses  are  not  included.  As  we  show 
below,  this  is  not  only  inconsistent  in  theory,  but 
also  causes  a  significant  bias  of  cost-effectiveness 
analyses  in  practice. 

The  statement  that  the  exclusion  of  non-health 
care  costs  is  acceptable  if  these  costs  add  a  constant 
cost  to  each  year  relates  to  a  theoretical  finding  by 
Garber  and  Phelps15  that  if  future  costs  are  condi¬ 
tionally  independent  in  the  sense  that,  conditional 
on  a  person’s  surviving,  the  costs  do  not  vary  with 
the  medical  intervention  in  question,  then  including 
those  costs  will  only  add  a  constant  to  the  cost- 
effectiveness  ratio.  Even  if  conditional  independence 
holds,  it  is  important  to  note  (as  does  the  Panel 
itself 2p  46 )  that  this  result  holds  only  when  compar¬ 
ing  interventions  at  a  single  age,  since  future  costs 
due  to  a  decrease  in  mortality  will  vary  widely  de¬ 
pending  on  age.12  For  example,  a  recent  study  of  hy¬ 
pertension  treatment  demonstrated  that  the  correct 
inclusion  of  these  costs  would  not  significantly  affect 
the  cost  per  QALY  gained  for  younger  men  and 
women,  but  would  increase  the  cost  per  QALY 
gained  by  nearly  $30,000  for  older  men  and 
women.14 

Moreover,  even  when  comparing  interventions  at 
a  single  age  and  when  future  costs  are  “condition¬ 
ally  independent,”  the  argument  turns  out  to  be  in¬ 
correct.  A  simple  example  can  illustrate  this.  Imag¬ 
ine  we  wish  to  compare  the  cost-effectiveness  of 
two  completely  independent  interventions  (A  and  B) 
for  two  completely  separate  but  both  otherwise  fatal 
diseases  at  a  single  age.  Assume  that  each  generates 
one  QALY,  but  that  A  adds  one  year  of  life  at  QOL 
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“  1  and  B  adds  two  years  of  life  at  QOL  =  0.5.  Thus 
intervention  A  has  bigger  effects  on  quality  of  life 
than  on  length  of  life,  while  intervention  B  has  big¬ 
ger  effects  on  length  of  life  than  on  quality  of  life. 
Assume  that  the  immediate  related  costs  are  $5,000 
for  A  and  $1,000  for  B.  Let  annual  related  costs  be 
zero  and  annual  unrelated  costs  (of  consumption 
minus  production)  be  $20,000  for  both.  Thus,  the 
definition  of  conditional  independence  is  met.  Ex¬ 
cluding  future  unrelated  costs,  C/E(A)  =  $5,000/ 
QALY  and  C/E(B)  =  $1,000/QALY.  Including  future 
costs,  the  cost-effectiveness  ratios  are  ($5,000  + 
$20,000)/1  QALY  =  $25,000/QALY  for  A  and  ($1,000  + 
2  *$20, 0001/1  QALY  =  $41,000/QALY  for  B.  Therefore, 
including  future  costs  changes  both  the  absolute 
and  the  relative  cost-effectiveness  of  the  two  inter¬ 
ventions  even  when  the  strict  definition  of  condi¬ 
tional  independence  is  met.  Thus,  in  contradiction 
to  the  findings  of  the  Panel,  the  correct  inclusion  of 
future  cost  is  not  an  empirical  question  depending 
on  whether  future  costs  add  a  constant  to  costs  in 
each  year.  This  example  also  illustrates  the  more 
general  point  that  excluding  future  costs  biases 
cost-effectiveness  analyses  of  interventions  among 
groups  with  positive  net  future  resource  use  (such 
as  the  elderly)  to  favor  interventions  that  extend  life 
over  interventions  that  improve  the  quality  of  life.12 
Conversely,  excluding  future  costs  among  groups 
with  negative  future  net  resource  use  (such  as  the 
young)  will  favor  interventions  that  improve  quality 
of  life  over  those  that  increase  length  of  life.  Esti¬ 
mates  of  the  magnitude  of  these  effects  of  including 
future  costs  suggests  that  they  are  large  enough  to 
significantly  alter  the  relative  cost-effectiveness  of 
interventions  such  as  the  treatment  of  hypertension, 
chemotherapy  for  colon  cancer,  and  hemodialysis 
for  end-stage  renal  disease.12 

Therefore,  the  recommendation  in  the  second 
paragraph  in  the  section  of  the  report  cited  above 
that  suggests  that  analysts  can  use  their  discretion 
about  the  inclusion  of  future  unrelated  costs  in  the 
Reference  Case  is  not  consistent  with  either  theo¬ 
retical  concerns  or  strongly  defensible  practical 
considerations.  It  also  leaves  the  literature  open  to 
being  saddled  with  one  set  of  analyses  that  do  not 
include  future  costs  and  are  not  comparable  to 
those  that  do  include  them.  The  recommendation 
that  a  sensitivity  analysis  be  conducted  if  these  ef¬ 
fects  are  large  begs  the  question,  since  it  does  not 
describe  how  one  is  to  know  whether  future  costs 
are  important  without  performing  such  an  analysis. 
Likewise,  the  rule  of  reason  argument  begs  the 
question,  since  it  does  not  define  what  constitutes  a 
significant  difference. 

For  these  reasons,  both  theoretical  and  empirical, 
it  seems  to  us  most  appropriate  to  include  future 
costs  in  the  Reference  Case.  If  a  sensitivity  analysis 
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suggests  that  their  effect  is  trivial,  as  might  be  ex¬ 
pected  when  there  is  no  or  almost  no  effect  on  sur¬ 
vival,  then  omitting  these  costs  may  not  be  problem¬ 
atic.  However,  an  adequate  sensitivity  analysis 
should  always  precede  the  decision  to  omit  such 
costs,  and  the  decision  to  omit  those  costs  should 
be  made  only  when  the  resulting  change  in  the 
cost-effectiveness  ratio  is  so  small  as  to  be  insignif¬ 
icant  for  any  sort  of  statements  of  relative  or  abso¬ 
lute  cost-effectiveness.  How  small  is  small  here  de¬ 
pends  on  the  importance  one  places  on  small 
differences  in  cost-effectiveness  ratios. 

Conclusion 

We  fully  agree  with  the  Panel  that  cost-effective¬ 
ness  analyses  should  be  carried  out  from  a  societal 
perspective,  including  all  costs  and  benefits  no  mat¬ 
ter  to  whom  they  accrue.  Furthermore,  we  applaud 
their  efforts  to  move  cost-effectiveness  analysis 
closer  towards  that  important  objective. 

Unfortunately,  several  of  the  recommendations  of 
the  Panel  concerning  costs  are  not  fully  consistent 
with  a  societal  perspective.  In  the  case  of  morbidity 
costs,  the  Panel's  recommendation  to  exclude 
changes  in  production  from  costs  is  likely  to  lead  to 
underestimating  the  cost-effectiveness  of  interven¬ 
tions  that  enhance  productivity  by  decreasing  mor¬ 
bidity.  For  the  reasons  discussed  above,  we  believe 
this  would  be  best  addressed  by  instructing  individ¬ 
uals  to  provide  QALY  weights  based  on  the  assump¬ 
tion  that  their  financial  circumstances  would  not 
vary  with  health  status  (i.e.,  that  they  have  paid  sick 
leave  and  do  not  pay  anything  out  of  pocket  for 
health  care  costs),  but  that  their  non-market  pro¬ 
ductivity  would  vaiy  with  health  status.  Further  em¬ 
pirical  work  is  required  to  determine  what  ap¬ 
proaches  to  quality-of-life  assessment  and  cost 
measurement  can  best  address  this  problem,  and 
we  feel  strongly  that  such  work  should  have  a  high 
priority. 

Similarly,  the  Panel's  recommendations  concern¬ 
ing  mortality  costs  will  substantially  distort  the  com¬ 
parison  of  cost-effectiveness  across  different  ages 
and  tend  to  inappropriately  favor  programs  that  ex¬ 
tend  life  over  programs  that  improve  the  quality  of 
life.  To  use  a  societal  perspective  for  cost-effective¬ 
ness  analysis,  it  is  necessary  to  include  consumption 
minus  production  during  life  years  gained  as  a  cost 
in  the  analysis.  While  including  such  costs  may 
sometimes  not  make  a  large  difference,  there  is 
good  evidence  that  they  may  also  commonly  make 
a  substantial  difference.  We  therefore  propose  that 
a  sensitivity  analysis  be  done  to  assess  the  magnitude 
of  future  costs  before  a  decision  is  made  to  omit 
them  in  a  particular  analysis. 

One  issue  we  have  neglected  to  this  point  is  the 
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political  acceptability  of  including  future  costs.  Im¬ 
plicitly,  there  may  be  a  feeling  that  this  will  result  in 
denial  of  care  to  the  elderly,  and  that  this  is  some¬ 
how  “unfair."  We  are  sympathetic  to  these  con¬ 
cerns,  but  return  to  the  fact  that  these  costs  are  real, 
and  note  that  the  vast  majority  of  cost-effective  in¬ 
terventions  among  the  elderly  will  continue  to  be 
cost-effective  when  future  costs  are  included,  and 
that  failing  to  account  for  future  costs  biases  cost- 
effectiveness  analyses  to  favor  interventions  that  ex¬ 
tend  life  over  interventions  that  improve  the  quality 
of  life.  If  QALYs  do  not  result  in  a  distributional  out¬ 
come  that  is  desirable  to  society,  we  propose  cre¬ 
ating  models  that  are  explicitly  based  on  those  val¬ 
ues  and  then  performing  analyses  to  identify 
interventions  that  do  help  realize  them.  If  there  are 
costs  or  benefits  we  have  failed  to  model  in  cost- 
effectiveness  analyses,  we  favor  modeling  them  ex¬ 
plicitly  as  well.  The  alternative  approach  of  ignoring 
costs  we  know  to  be  real  tends  only  to  create  addi¬ 
tional  biases,  such  as  favoring  length  of  life  over 
quality  of  life  among  the  elderly. 

The  work  of  the  Panel  on  cost-effectiveness  in 
Health  and  Medicine  is  a  useful  step  in  the  meth- 
odologic  development  of  medical  cost-effectiveness 
analysis.  Nevertheless,  the  presence  of  these  impor¬ 
tant  inconsistencies  in  the  Panel’s  recommendations 
concerning  costs  demonstrates  the  need  to  continue 
to  evaluate  the  methods  of  cost-effectiveness  anal¬ 
ysis  if  the  field  is  to  realize  its  promise  to  improve 
the  allocation  of  scarce  resources  for  health  care.  As 
the  Report  of  the  Panel  effectively  argues,  this  will 
surely  require  attention  to  both  the  theoretical  foun¬ 
dations  of  cost-effectiveness  analysis  and  the  sim¬ 
plicity,  feasibility,  and  broad  acceptance  of  its  meth¬ 
ods.  It  is  possible  that  some  of  the  Panel's 
conclusions  on  these  issues  may  have  been  shaped 
by  the  latter  factors,  in  which  case  it  is  our  hope 
that  these  issues  may  come  to  be  reconsidered  over 
time  as  methodologic  advances  in  areas  such  as 
quality-of-life  assessment  and  estimation  of  future 
costs  make  us  more  able  to  address  these  issues 
successfully.  Indeed,  the  need  for  ongoing,  critical 
appraisal  of  the  methods  of  cost-effectiveness  anal¬ 
ysis  is  reinforced  by  the  fact  that  cost-effectiveness 
analysis  is  by  no  means  the  only  approach  to  allo¬ 
cation  of  resources  within  health  care.  Though  they 
also  have  their  own  limitations,  cost-benefit  analy¬ 
ses  that  assess  willingness  to  pay,  formal  and  infor¬ 
mal  reflections  of  expert  and  public  opinion,  and 
greater  attention  to  the  development  of  the  knowl¬ 
edge  and  incentive  systems  to  help  patients  and 
their  physicians  make  better  decisions  concerning 
medical  spending  are  all  alternatives  worthy  of  con¬ 
sideration.  The  presence  of  these  important  alter¬ 
natives  to  cost-effectiveness  analysis  argues  strongly 
for  persistence  in  questioning  and  attempting  to  ad¬ 
vance  its  theoretical  foundations. 
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Response 

•  On  the  Role  of  Theory  in  Cost-Effectiveness  Analysis — 
A  Response  to  Garber,  Russell,  and  Weinstein 

DAVID  MELTZER,  MD,  PhD,  MAGNUS  JOHANNESSON,  PhD 


The  comments  of  Garber,  Russell,  and  Weinstein  in 
response  to  our  paper  add  to  the  important  contri¬ 
butions  to  cost-effectiveness  analysis  that  have 
grown  out  of  the  work  of  the  Panel  on  Cost-Effec¬ 
tiveness  Analysis  in  Health  and  Medicine.1”3  Al¬ 
though  they  differ  from  each  other  and  from  our 
work  in  some  areas,  there  appear  to  us  to  be  many 
areas  of  agreement  and  important  opportunities  to 
further  advance  the  field  that  come  from  the 
exchange  among  us. 

A  first  area  of  agreement  is  that,  to  the  extent  that 
welfare  economics  is  to  serve  as  a  basis  for  cost- 
effectiveness  analysis,  all  future  medical  and  non¬ 
medical  costs  resulting  from  increased  survival 
should  be  included  and  measures  of  quality  of  life 
and  costs  should  be  designed  to  ensure  that  all  ben¬ 
efits  and  costs  are  neither  omitted  nor  counted 
more  than  once.  Indeed,  the  Panel’s  report  empha¬ 
sizes  the  value  of  welfare  economics  as  a  “logical” 
foundation  for  cost-effectiveness  analysis.4  We  agree 
with  this  viewpoint,  because  welfare  economics  is 
based  fundamentally  on  the  principle  of  using  lim¬ 
ited  societal  resources  to  maximize  social  welfare 
and  because  it  encourages  the  explicit  consideration 
of  both  objectives  and  constraints.  A  second  area  of 
agreement  is  that  a  variety  of  practical  and  ethical 
concerns  were,  and  should  have  been,  factored  into 
the  Panel’s  recommendations  about  these  issues,  so 
that  consistency  with  welfare  economics  should  not 
be  the  sole  consideration  by  which  the  Panel’s  rec¬ 
ommendations  are  assessed.  The  heart  of  the  issues 
raised  in  our  exchange  is  about  balancing  these  the¬ 
oretical,  practical,  and  ethical  concerns. 

With  regard  to  the  inclusion  of  productivity  and 
other  costs  as  part  of  quality-of-life  measures,  we 
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appear  to  be  in  broad  agreement  that  there  is  am¬ 
biguity  about  costs  in  all  existing  quality-of-life  mea¬ 
sures  (except  the  HUI).  Moreover,  we  appear  to 
agree  that,  despite  the  temptation  on  practical 
grounds  to  use  existing  measures  without  alteration, 
this  ambiguity  requires  clarification  both  in  posing 
quality-of-life  questions  and  in  measuring  costs  to 
ensure  that  all  quantitatively  important  costs  are  nei¬ 
ther  omitted  nor  counted  more  than  once.  While  we 
agree  with  the  need  to  better  quantify  the  magnitude 
of  productivity  losses  not  captured  by  present  prac¬ 
tices,  we  remain  convinced  that  the  substantial  pay¬ 
roll  and  income  taxes,  as  well  as  public  or  private 
disability  insurance  in  most  countries,  make  it  ex¬ 
tremely  likely  that  such  omissions  are  often  a  large 
part  of  total  productivity  losses.  For  this  reason,  we 
are  pleased  that  at  least  two  of  our  respondents 
agree  with  us  that  these  concerns  will  likely  be  ad¬ 
equately  addressed  only  by  using  measures  that  ex¬ 
plicitly  instruct  respondents  not  to  consider  the  per¬ 
sonal  economic  consequences  of  their  illness  and  to 
include  the  full  productivity  costs  of  morbidity  in  the 
numerator  of  the  CE  ratio.  Testing  and  validating 
quality-of-life  measures  that  meet  these  criteria  is  an 
important  area  for  future  work. 

Only  Garber  and  Weinstein  respond  specifically  to 
the  future-costs  issue.  Garber  emphasizes  the  im¬ 
portance  of  measuring  all  future  health  and  non¬ 
health  costs  unless  they  are  small;  we  again  note 
that  the  effects  of  including  these  costs  will  be  small 
only  under  rather  strict  theoretical  conditions  and 
that  actual  calculations  of  the  effects  of  including 
future  costs  show  that  such  costs  often  cause  sub¬ 
stantial  changes  in  cost-effectiveness  ratios.S6  We 
should  also  mention  in  response  to  Garber  that  the 
theoretical  result  about  the  need  to  include  future 
costs  that  we  illustrate  with  our  example  does  not 
require  the  discontinuities  in  our  simplified  exam¬ 
ple;  the  same  result  is  present  in  the  continuous 
model  analyzed  in  Meltzer’s  original  paper  on  future 
costs.5 

Weinstein  objects  to  the  inclusion  of  future  non¬ 
medical  costs  of  life  extension  on  the  grounds  that 
this  will  reflect  the  current  distribution  of  income 
and  wealth  in  society  and  thus  “be  laden  with  some 
of  the  most  objectionable  features  of  CBA  which  im- 
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pede  its  more  widespread  acceptance/'  We  note  in 
response  that,  as  Weinstein  recognizes,  including 
such  costs  is  conceptually  no  different  than  using 
wages  to  value  caregiver  time,  which  the  Panel  en¬ 
dorses,  and  moreover  that  there  is  no  need  to  use 
an  individual’s  personal  wage  rate  in  such  calcula¬ 
tions,  since  one  can  use  a  population  average  if  one 
wants  differences  in  the  distribution  of  wealth  not 
to  affect  such  calculations.  In  principle,  distribu¬ 
tional  concerns  could  also  be  incorporated  into 
cost-effectiveness  analysis  by  weighting  costs  and 
QALYs  differently  in  different  population  groups.  It 
is,  however,  hard  to  see  the  logic  of  any  weighting 
scheme  that  would  inherently  value  a  dollar  spent 
due  to  extension  of  life  differently  than  a  dollar  spent 
any  other  way.  Moreover,  we  note  that  omission  of 
future  costs  results  in  spending  too  much  on  inter¬ 
ventions  that  extend  life  relative  to  those  that  im¬ 
prove  quality  of  life  in  the  elderly.  In  conclusion,  we 
find  the  arguments  for  excluding  future  costs  on 


ethical  or  practical  grounds  unconvincing  and  rec¬ 
ommend  the  inclusion  of  the  full  future  costs  in 
cost-effectiveness  analysis. 
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Do  Quality-Adjusted  Life  Years  Reflect  Patient  Preferences?  Validation  Using 
Revealed  Preference  for  Intensive  Treatment  of  Insulin-Dependent  Diabetes 
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DO  QUALITY-ADJUSTED  LIFE  YEARS  REFLECT  PATIENT  PREFERENCES?  VALIDATION 
USING  REVEALED  PREFERENCE  FOR  INTENSIVE  TREATMENT  OF  INSULIN-DEPENDENT 
DIABETES  MELLITUS.  DO  Meltzer,  TS  Polonsky.  University  of  Chicago,  Chicago,  IL. 

Purpose:  Quality-adjusted  life  years  (QALYs)  are  often  used  to  evaluate  therapies  in  medical  cost- 
effectiveness  analysis,  but  their  validity  as  a  measure  of  patient  preferences  is  unproven.  The  absence  of  an 
accepted  measure  of  patient  preferences  has  hindered  efforts  to  validate  QALYs.  We  used  the  clinically 
accepted  standard  for  identifying  the  preferences  of  patients  -  their  informed  choices  -  to  assess  the 
“revealed”  preferences  of  130  patients  with  IDDM  for  intensive  (INT)  vs.  conventional  (CONV)  therapy. 
To  assess  the  validity  of  QALYs  as  a  measure  of  patient  preferences,  we  then  used  standard  techniques  to 
calculate  the  change  in  QALYs  with  INT  and  examined  whether  this  predicts  patient  choice  of  therapy. 

Methods:  Time  trade-off  questions  were  used  to  measure  patient  beliefs  about  quality  of  life  (QOL)  with 
each  therapy  and  the  major  complications  of  IDDM.  Frequency  assessment  questions  were  used  to  assess 
patient  beliefs  about  the  efficacy  of  INT.  Questions  about  preference  for  the  present  vs.  future  were  used  to 
assess  discount  rates.  A  decision  model  based  on  the  Diabetes  Control  and  Complications  Trial  (DCCT) 
was  used  to  calculate  the  change  in  QALYs  with  INT. 

Results:  Patients  on  CONV  reported  lower  QOL  with  INT  vs.  CONV  (p<0.01),  while  patients  on  INT 
reported  no  difference  in  QOL.  However,  patients  on  INT  and  CONV  did  not  differ  in  the  QOL  they 
associated  with  complications  of  IDDM.  Patients  on  INT  reported  higher  estimates  of  efficacy  of  INT  in 
preventing  complications  than  did  patients  on  CONV  (p<0.0l),  but ‘overestimated  the  benefits  of  INT 
compared  to  the  results  of  the  DCCT.  Patients  on  INT  also  exhibited  lower  discount  rates  than  did  patients 
on  CONV.  The  change  in  QALYs  predicted  reported  treatment,  with  an  area  under  the  ROC  curve  (AUC) 
of  0.84.  Change  in  QALYs  also  predicted  the  behaviors  that  define  INT,  but  the  AUC  was  only  0.73, 
perhaps  because  the  correlation  of  QALYs  and  reported  therapy  is  enhanced  by  attempts  to  minimize 
cognitive  dissonance. 

Conclusions:  QALYs  correlate  with  patient  preferences  as  revealed  by  their  choices.  However,  the 
possibility  that  answers  reflect  patient  efforts  to  minimize  cognitive  dissonance  concerning  their  choice  of 
therapy  suggests  that  this  should  be  considered  an  upper  bound  on  the  extent  to  which  QALYs  reflect 
patient  preferences.  Analyses  whose  results  depend  on  measurement  of  QALYs  should  be  interpreted  with 
caution.  Revealed  preference  techniques  should  be  used  to  validate  and  improve  QOL  assessment. 
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Structure  and  Early  Results  of  a  Prostate  Cancer  Decision  Model  I 

David  O.  Meltzer.  Anirban  Basu,  Brian  Egleston.  University  of  Chicago,  Chicago,  IL.  , 

|  Purpose:  To  develop  an  innovative  decision  model  to  assess  the  effectiveness  and  cost-effectiveness  of  | 
.alternate  treatment  strategies  on  men  with  clinically  localized  prostate  cancer  (PC), 
s  Methods:  We  reviewed  all  existing  decision  models  of  PC  screening  and/or  treatment.  We  then  .  j 

ideveloped  a  new  model  based  on  pathologic  stage  and.grade.of  PC  that  addresses'many-important-aspecfo  | 
of  the  natural  history  of  prostate  cancer  progression  neglficled.byjiiej'ious  analysis.  Key  features  include:  i 
'modeling  of  tumor  heterogeneity  and  progression  rates  by  grade,  misclassification  of  tumor  stage  due  to  J 
.discrepancy  between  clinical  and  pathological  stage,  and  ability  of  screening  and  treatment  to  affect 
prevalence  of  PC  in  the  undetected  population,  and  accounting  for  the  effect  of  benign  prostatic  ; 

hypertrophy  and  its  management  on  the  detection  of  prostate  cancer.  The  core  of  the  model  consist  of  a  j 
•Markov  process  defined  by  a  state  vector  containing  1440  elements  spanning  durable  characteristics  of  PC 
such  as  detectability.  The  model  is  then  run  from  age  50  to  99  years  to  assess  the  effects  of  screening  and  ! 
'  treatment  policies.  Model  parameters  are  derived  by  extensive  literature  review  and  validated  using  j 
!  aggregate  incidence  and  mortality  data  as  well  as  individual  level  data  using  maximum  likelihood  ! 

techniques.  In  this  abstract  we  report  early  analyses  of  the  effects  of  alternative  treatments  on  outcomes  by{ 
tumor  grade  (well,  moderate  and  poorly  differentiated)  for  men  50  to  80  years  of  age.  ’ 

Results:  Treatment  offers  only  modest  increase  (maximum  0.5  years)  in  non-discounted  life  expectancy  : 
for  well-differentiated  tumors  at  all  ages.  Effects  are  more  favorable  (>  1  year)  for  moderately  and  poorly  • 
differentiated  tumors,  but  only  for  men  below  age  70.  Treatment  was  cost-effective  for  men  in  this  j 

category  (CER  $40K  -  $76K  per  non-discounted  life  year).  When  quality  of  life  weights  were  added, 
quality-adjusted  life  expectancy  (QALE)  was  reduced.  QALE  increased  only  in  extreme  sensitivity  j 
analyses  that  ignored  age-specific  decrements  in  QOL,  complications  of  treatment,  or  discounting.  Even  1 
;  in  these  extreme  cases,  treatment  was  at  best  marginally  cost-effective  by  the  $100,000  per  QALY 
standard. 

Conclusions:  The  preliminary  results  of  our  model  suggest  that  radical  prostatectomy  and  radiation  • 
therapy  improve  non-discounted  life  expectancy  for  younger  patients  with  more  advanced  tumors  at  a  cost  j 
of  less  than  $100,000  per  life  year  saved.  Similar  benefits  are  not  evident  in  QALE,  but  the  results  may  be 
sensitive  to  assumptions  about  discount  rates  or  QOL  weights.  I 


